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ABSTRACT 

This reports presents interview findings with education 
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contained the following groups: superintendents and other local education 
officials; chief state school officers; state higher education executive 
officers; state legislators; governors’ educational policy advisors; 
congressional staff members; and education association executive directors. 
Altogether, 142 interviews were conducted. The interviews covered the 
following aspects of educational research: (1) research priority areas from 

policymakers' perspectives; (2) access to and use of existing research; (3) 
assessment of existing research; and (4) suggestions for improving education 
research. Student achievement--especially achievement in reading, math, and 
science--and its relationship to effective instruction, curricula, programs,- 
and policies was the highest priority for research shared by the policymakers 
interviewed across groups. Respondents also expressed the importance of 
research in teacher development and quality; assessment and accountability; 
educational finance; and achievement gaps. Among major policy interests, 
respondents cited the financing of education as their major concern. Appended 
are the study's methodology, interview protocol and instrument, a sample 
introductory letter, and counts by frequency of reading research reports. 
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EXECUTIVE SUMMARY 



The Institute of Education Sciences (IES, formerly the Office of Educational Research 
and Improvement, or OERI) has begun an effort to gather input directly from education 
policymakers. Synectics for Management Decisions, Inc. (Synectics) worked with IES to 
plan, design, and conduct interviews with education policymakers. This report presents 
the interview findings and documents the project design and procedures in appendices. 

The project used a purposive sampling approach to select policymakers from the 
following groups: 

• Superintendents and other local education officials; 

• Chief state school officers (CSSOs); 

• State higher education executive officers (SHEEOs); 

• State legislators; 

• Governors’ educational policy advisors (GEPAs); 

• Congressional staff members (including staff members of the Senate Committee 
on Labor and Human Resources and the House Committee on Education and the 
Workforce, and educational policy specialists on the staffs of members of both 
houses); and 

• Education association executive directors. 

The interviews covered the following aspects of education research: research priority 
areas from policymakers’ perspectives; access to and use of existing research; assessment 
of existing research; and suggestions for improving education research. The findings in 
this report may provide the Director of the Institute of Education Sciences with 
information he can use to “increase the relevance of [the Department of Education’s] 
research in order to meet the needs of . . . customers” (Objective 4.2, Department of 
Education’s Strategic Plan 2002-2007). 

Findings from the interviews are summarized below. The responses to question 1, which 
asked respondents to name the two highest priority areas where they thought further 
research was most needed, are followed by the responses to question 5, which asked 
respondents to name other high priority issues, and, in turn, by responses to questions 2, 

3, and 4. 

Highest Priority Issues In Need of Further Research (Question 1) 

Effective Practices, Programs, and Policies 

Student improvement, especially achievement in reading, math, and science, and its 
relationship to effective instruction, curricula, programs, and policies was the highest 
priority for research shared by the policymakers interviewed across groups. The 
respondent groups, however, expressed their priorities differently, and often related them 
to diverse aspects of the education system. For example, the superintendents and other 
local education officials interviewed frequently discussed student learning together with 
instruction and curriculum research and most of them saw instruction and curriculum 
research as the highest priority for research. The state- level policymakers — CSSOs, 
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SHEEOs, state legislators, and GEPAs — and the education association executive 
directors who were interviewed tended to address the issue in relation to teacher training, 
school intervention programs, and standards-based testing. The Congressional staff 
members interviewed, on the other hand, highlighted teacher quality and the development 
of a knowledge base in connection to achievement and performance. 

Teacher Development and Quality 

Teacher quality-related issues, including teacher preparation, recruitment, and 
professional development, was another high priority area widely shared by respondents. 
Many respondents in each policymaker group (except the SHEEOs) raised concerns 
about teacher quality and supply as a key issue for research. Only one SHEEO mentioned 
teacher education in relation to other issues. 

Assessment and Accountability 

Standards-based assessment and accountability was seen as a highest research priority by 
a large number of respondents across groups, although there were different opinions 
about its value. Some respondents called for studies to examine the rationale and 
empirical consequences of standards and assessments in connection to the achievement 
gap and school dropout. The issue was not mentioned by state higher education executive 
officers interviewed. 

Education Finance 

Education finance was mentioned as another highest research priority by many 
respondents, though from various perspectives across policymaker groups. 
Superintendents and local education officials and education association executive 
directors were keenly concerned about financial issues and their impact on achievement, 
class size, teacher recruitment, and competition with alternative schools. The SHEEOs 
interviewed were interested in studies on the affordability and quality of higher education 
due to changing fiscal conditions. The state legislators interviewed had broader views of 
school financing in relation to equity and the No Child Left Behind legislation. 
Congressional staff and GEPAs did not address financial issues in their interviews. 

Close Achievement Gaps 

Achievement gaps relating to various sociodemographic groups was another high 
research priority concerning many of the superintendents and other local education 
officials, CSSOs, and Congressional staff. A number of high priorities were proposed by 
only one or two policymaker groups, and these issues seemed to reflect the group’s 
unique work scope. For example, instruction and curriculum research was seen as a 
highest priority by 13 out of the 30 superintendents and local education officials who 
were interviewed. This issue interested respondents in this group more than any other 
issue, but it was rarely emphasized by respondents in the other groups. Interest in 
research on rural-urban disparities was voiced by CSSOs. 
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Other High Priority Issues (Question 5) 

Additional high priorities for research were proposed by respondents were far more 
diverse than the highest priorities they discussed. Few issues were shared across groups 
or even within a group. 

• Basic research issues: The conceptual clarification of scientifically-based 
methodology (by a superintendent and a SHEEO), comparison of experimental 
design-based studies vis-a-vis other research approaches, brain science in 
connection with learning theories (by a CSSO and an association executive), and 
cross-disciplinary syntheses of research for education applications. 

• Practice-logistic problems: School schedules and bus schedules, year-round 
schools, crowding in schools, and alternative schools such as online schools, 
voucher programs, and charter schools were mostly interesting to superintendents 
and local education officials. 

• Emerging issues: Using new approaches to education research (e.g., economic 
theories) and examining issues such as small high schools, extending the high 
school senior year into junior college, attracting minorities to teaching, 
developing guidelines or benchmarks to determine the effectiveness of state 
programs, and focusing on successes of students and schools rather than 
disadvantages. 

• Enduring issues : Special education, the use of technology in education, how 
students and their families perceive affordability and college selection, student 
mobility in higher education (by a SHEEO and an association executive), the 
world of work from the perspectives of student preparation and state and national 
economic developments, vocational education, arts education, science and 
technology education, high school and dropouts and older students, and English 
language learning. 

Use of Education Research Resources (Question 2) 

More than half of the respondents (49 out of 71) reported that they read research reports 
“most of the time” or “just about always,” and at least a half of the respondents in each 
policymaker group read research reports that frequently, except for the state legislators, 
both of whom read these reports “only some of the time.” 

Most-used Sources 

The following sources were fairly widely cited by all the respondents: the Internet, ERIC, 
national and regional professional associations, professional conferences, journals and 
magazines, the federal government (specifically NCES and IES). Regional education labs 
and other regional/state education services were valued as good research resources by the 
interviewed superintendents and other local education officials and by some other state- 
level policymakers. Internal research staff and resources were a source for most of the 
policymakers interviewed, except for the superintendents and other local education 
officials (but three respondents from large districts also had in-house research resources). 

Criticism 

While sharply critical of the existing education research, most respondents acknowledged 
the value of research to their work. Their essential criticisms included: 
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• Overly theoretical and academic orientation'. This was a shared concern, but 
superintendents and other local education officials were particularly adamant 
about this being a problem. 

• Gaps in knowledge and lack of scientific rigor in studies'. State- level 
policymakers, Congressional staffers, and national association executives laid 
stress on this weakness; while some local education officials reserved judgment. 

• Political or marketing bias and contamination'. Many respondents from each 
policymaker group complained about this. 

• Lack of detailed information breakdowns by geographic and demographics: 
Many respondents, especially superintendents and state- level policymakers were 
dissatisfied by the existing statistics and survey data; they said national data had 
limited local use. 

Most respondents pointed out that lack of time to filter the available information made it 
difficult for them to use research; they called for research summaries that were tailored to 
different user groups and disseminated through a variety of channels. 

Steps the Department Can Take to Improve Education Research 
(Question 3) 

The respondents, whether at the district, state, or national level, shared strikingly similar 
views regarding steps the Department could take to make education research more useful, 
accessible, and relevant. They underscored the importance of ongoing dialogue with the 
Department and feeling confident that their needs and concerns were being heard by 
senior Department officials. A number of respondents highlighted the importance of 
ensuring that Department research reports were free of political bias. 

Strengthen Services via Internet 

These policymakers sought help in quickly identifying useful education research and 
communicating it to their staffs. The Internet was a preferred tool for rapid information 
dissemination, whether through e-mail or Web sites. They also suggested the Department 
simplify access to its Web pages and to ERIC and improve the indexing and searching 
tools for these resources. 

Summary of Research 

There was nearly unanimous agreement among the policymakers interviewed that 
summaries, highlights, and abstracts of education research were indispensable tools for 
education policymakers and that the Department should play a leading role in making 
these tools available. A number of respondents proposed that the Department provide 
links from report summaries to the complete research reports. 

End-user Oriented Research Syntheses 

A number of the education policymakers interviewed called on the Department to 
translate the language of academic education research into usable guidance for practical 
decision-making. These respondents felt it was important to keep the needs of “end 
users” in mind, look at issues from diverse perspectives, and address fundamental 
questions related to practice. These policymakers further proposed that the Department 
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provide syntheses of research findings and assistance in resolving conflicting research 
evidence through meta- analyses on specific research issues. They also suggested the 
Department try to expedite release of its education research findings: multi-year delays in 
publication of NCES and other data hampered effective decision-making. 

Expanding Information Dissemination and Sharing 
State- and local- level policymakers particularly underscored the unique role of the 
Department in “aggregating research and resources” by collecting and disseminating 
district and state education research across the country, and called for expanded efforts in 
this area, as well as in facilitating information sharing among states and local agencies. 

Respondents additionally called for the Department to expand its role as an information 
broker by providing more information on education research funded by foundations and 
academic and private sector research institutes. 

Training in Use of Research 

Superintendents underscored the need for the Department to provide training at the 
district level in the interpretation and use of education research. 

Major Policy Interests (Question 4) 

Broad Interest in Education Finance 

While the respondents’ individual policy interests reflected their institutional priorities, as 
well as their personal intellectual concerns, several issues were cited across all the groups 
interviewed. The financing of education, whether prekindergarten, elementary/secondary, 
or postsecondary, was a major concern to most groups. Many respondents noted the 
difficulty of choosing among multiple priorities in allocating limited funds. They were 
also concerned about the relationship of federal financial resources to state and local 
funding. 

Assessing Performance 

Enhancing and assessing student achievement was a recurrent concern among 
respondents. Respondents were concerned with developing performance measures, 
enabling them to better assess program effectiveness and school and college quality. 

Issues of teacher quality and development of educational leaders were of importance. 

Education Reform, Understanding Connectivity 

Policymakers were interested in education reform initiatives. They sought, as well, to 
establish linkages across the educational system, understand the connections between 
education and the world of work, and discern the impact of education on economic 
development. Educational technology and its role in teaching and learning attracted the 
interest of several respondents. 

Other Areas of Concern 

Access to education was important, including ensuring the affordability and availability 
of early childhood education. Respondents also focused on enhancing English- language 
learning for students for whom English was not a first language. Superintendents and 
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other local education officials were particularly concerned with nuts and bolts questions 
of school operations and facilities management. Flexible funding and decentralized 
decision-making were important to them, as well. 

Use and Quality of Research 

Addressing the utility and quality of education research, respondents called for greater 
sharing of research information across the states, more comparative research, and greater 
research specificity and timeliness. While some of these policymakers found current 
education research of value, many others sought research more directly linked to practice 
or translated into clear, directly applicable guidance. While a number of respondents 
called for more rigorous scientifically- grounded education research, others underscored 
the importance of qualitative, more holistic approaches. 

Gaps in Research: The respondents identified gaps in existing research, citing such areas 
as rural education, education for poor children, and mathematics instruction. Respondents 
criticized the “bandwagon” effect in research, with a number of individuals expressing 
concern over the politicization of research and the use of education research to fit 
particular special- interest agendas. 

Other Research Models: Several policymakers expressed interest in the applicability to 
the field of education of models and perspectives from outside the field. They found 
research from the world of business and economics highly relevant and insightful in 
addressing issues of innovation and systemic change in education. 
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INTRODUCTION 

Background 

Synectics for Management Decisions, Inc. (Synectics), under the direction of the Institute 
of Education Sciences (IES, formerly the Office of Educational Research and 
Improvement, or OERI), U.S. Department of Education, conducted interviews with 
education policymakers in Congress, local school districts, state education agencies, and 
key national organizations that represent education decision makers. The project, and this 
report, provides the Director of the Institute of Education Sciences with information he 
can use to achieve the Department of Education’s Strategic Plan 2002-2007 Objective 
4.2, “Increase the relevance of our research in order to meet the needs of our customers.” 

Synectics worked with IES to plan, design, and conduct the interviews. This report 
presents the substantive findings from the interviews and documents the project design 
and procedures in appendices. 

Design and Procedures 

The project entailed a purposive sampling design, interview protocol development and 
pretests, mailing of introductory letters, telephone interviews of the selected 
policymakers across the country, documentation of the interviews, and writing a report. 

Purposive Sampling 

The target population was policymakers working in various jurisdictions, educational 
levels, and geographic and demographic categories. To maximize the relevance of their 
input, given the constraints of a limited project budget, IES decided on a purposive 
sample. Differing from random sampling, purposive sampling is not meant to produce 
quantitative information that represents a given population through statistical estimation. 
The largely qualitative information collected from a purposive sample may nevertheless 
convey policymakers’ perspectives with a reasonable depth and inclusiveness. 

In employing this approach, the task team worked with the IES staff to select the most 
influential policymaking entities, while covering different aspects of public education, 
including different jurisdictions, educational levels, and regional and demographic 
characteristics. A sample of the following groups of policymakers were included in this 
study (see appendix A for details of sample development): 

• Superintendents and other local education officials; 

• Chief state school officers (CSSOs); 

• State higher education executive officers (SHEEOs); 

• State legislators; 

• Governors’ educational policy advisors (GEPAs); 

• Congressional staff members (including staff members of the Senate Committee 
on Labor and Human Resources and the House Committee on Education and the 
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Workforce, and educational policy specialists on the staffs of members of both 
houses); and 

• Education association executive directors. 

The distribution of types of policymakers in the sample is shown in table 1. 



Table 1. Final sample members and completed interviews, by categories of 
policymakers 



Policymaker group 


Number of selected 
respondents 


Number of completed 
interviews 


TOTAL 


79 


71 


Superintendents and other local education 


officials 


34 


30 


Chief state school officers (CSSOs) 

State higher education executive officers 


10 


9 


(SHEEOs) 


10 


10 


State legislators 


2 


2 


Governors’ education policy advisors 


5 


4 


Congressional staff members 


*8 


*6 


Education association executive directors 


10 


10 



* The number of selected respondents included four staff members from the House and four from the 
Senate; three interviews were completed with staff members from each group. 



Local- and state-level policymakers were selected so that states and localities were 
represented across key geographic and demographic categories, including: Census region, 
district urban-rural locale, district enrollment size, state percent of urban population, state 
population size, and state math 4 th grade achievement level. Information sources used 
included the Census Bureau’s 2002 population estimation, the Common Core of Data 
(CCD) on district enrollment and locale, and the 2000 NAEP mathematics performance 
by states. See appendix table A- 2 for a summary of characteristics of the selected district 
and states. 

Substitutes were used when the sampled individual from the primary state or district was 
unavailable or unwilling to participate. The sample identified as respondents the top 
decision maker or a high-ranking staff member who led research and policy making in 
each selected agency. 

Interview Protocol 

The interview protocol was designed to be used in a telephone interview. To minimize 
the burden on the respondents, the protocol contained only five open-ended questions, in 
addition to an introduction. The five questions were as follows: 

1 . What, in your opinion, are the two highest priority areas in which further 
research is most needed? 

(a) Why is this so? 



IES Findings from Interviews with Education Policymakers 



Introduction 



Synectics for Management Decisions , Inc. 



3 



(b) And, when you describe these topics, what level of education are you 
mainly concerned with? 

(c) Of the priority areas you listed, which would you rank higher? 

2. When looking for information on effective educational programs or practices, 
do you read research studies or reports of evaluations of the programs you are 
interested in (a) never, (b) only some of the time, (c) most of the time, or (d) 
just about always? 

(a) When you have used research information, how have you obtained it? 

(b) What would make it easier for you to use research information on a 
regular basis? 

3. What could the U.S. Department of Education do to make education research 
more useful, more accessible, or relevant to your work? 

4. In terms of your own work in education, what are your major policy interests? 

(a) Has the research you’ve used been useful to you in addressing your 
specific areas of interest or providing fruitful guidance? 

(b) Could you tell me about your sense of the research you’ve used, both in 
terms of the amount of existing research and the quality of that research? 

5. Finally, on reflection, are there any other high priority issues, areas, or themes 
in American education in which you would like to see more, better, or a 
different type of research? 

See appendix B for the complete interview protocol. The protocol was developed through 
collaboration between Synectics and the EES staff It was twice pretested in the process. 

Pretests 

To ensure that both the interview protocol and the communications procedure were 
appropriate for gathering adequate information, Synectics conducted two pretests. A total 
of seven state policymakers, association directors, and Congressional staff were 
interviewed during the first test. Results of the first test were used to revise the interview 
protocol. The revised protocol was again tested with five policymakers, including one 
Congressional staff member, one state legislator, one association director, and two local 
district superintendents. The policymakers in the pretests were chosen from entities that 
were not included in the study sample. The pretest results were documented and used to 
revise the protocol and the contact procedure. 

Interview Procedures 

A letter was drafted to inform each of the selected policymakers about the purpose, 
content, and procedure of the planned interview. The letter also advised respondents that 
their participation was voluntary and their identification and responses were to be kept 
confidential. Upon approval and sign-off by the Director of the Institute of Education 
Sciences, the letter was sent to the selected decision makers a week before the telephone 
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calls began. (See appendix C for a copy of the letter.) The letters were faxed or e-mailed 
to respondents’ offices upon request. 

Two Synectics staff members who were familiar with the current education policy issues 
conducted all the interviews. An interview typically followed numerous initial calls and 
e-mails to make the arrangement, and lasted from 6 to 48 minutes, with an average of 
about 20 minutes to complete. 

Twenty sampled individuals designated other senior staff to respond {designees). Four 
sampled individuals declined to participate, requiring substitutes from similar entities to 
be selected {substitutes). Eight selected sampled members were either not reached or 
could not be scheduled for an interview within the study timeframe. The overall response 
rate was 90 percent. (See appendix table A-4 for detailed counts of respondents by 
policymaker groups.) The formal interviews began on October 4, 2002, and were 
completed by November 22. 

Documentation 

For each respondent, the following information was recorded in addition to their 
responses: the respondent’s name, official tide, affiliation, date and time of the interview, 
and the interviewer’s name. For interviews completed by designees or substitutes, the file 
also recorded the reason the sampled individual was replaced and the substitute’s official 
responsibilities relevant to the project. 

Coding of Responses to Question 1 (Highest Research Priorities) 

Asked to specify and rank two highest priority areas for research, some respondents 
proposed more than two issues or only one issue and many emphasized the equal 
importance and close association of the specified issues. The issues frequendy overlapped 
in respondents’ descriptions. For example, a respondent may have talked about teacher 
training for early childhood programs in reading and math or about high school dropout 
among at-risk students with low achievement. To organize the information gathered from 
the interviews, the responses were grouped into broad policy areas. Thus, the issues are 
sometimes presented together as the highest priorities without coding or specifying the 
ranking made by respondents. See appendix A for the approach to identifying and 
categorizing the high priority issues. 

Limitations 

Information generated from the purposive sample cannot be considered representative of 
education policymakers nationwide. Furthermore, a short telephone call to assess broad 
areas in many complicated aspects of education research restricts the scope and depth of 
the conversations. The findings in this report should serve as an illustrative source for 
federal research managers’ deliberation in forming new research priorities. 

Format of Report 

The remainder of the report is organized in seven sections corresponding to the seven 
groups of education policymakers interviewed in this study. Each section contains a 
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description of the findings organized by the five questions asked in the interview. The 
responses to question 1, which asked respondents to name the two highest priority areas 
where they thought further research was most needed, are followed by the responses to 
question 5, which asked respondents to name other high priority issues. The other 
questions appear in numerical order. In addition, appendix A contains methodological 
details of the study, appendix B contains the interview protocol, appendix C contains the 
introductory letter sent to each of the sampled individuals, and appendix D contains a 
table showing the frequency counts for reading research reports, by policymaker group. 
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SUPERINTENDENTS AND OTHER LOCAL EDUCATION 

OFFICIALS 

Superintendents and other local education officials 1 concerned with such areas as 
planning, research and evaluation, and assessment comprised the largest group of 
educational decision makers in the interviews. In interviews, they presented highly 
diverse and sometimes conflicting views regarding education research. Their perceived 
research priorities reflected distinct or even rival local interests, needs, and perspectives. 

The extent to which they accessed and utilized research information varied substantially. 
Nevertheless, some common themes did emerge in the analysis and may serve as 
indicators for readjusting the national research agenda. Even the dissidence in the group 
is useful in revealing the complex reality of education practice and education research. 

Highest Priority Issues in Need of Further Research 

(Question 1) 

Local education specialists proposed the following as the highest priority issues in need 
of further research: curriculum and instruction research (16 respondents), assessment and 
accountability (10 respondents), achievement gaps (1 1 respondents), early childhood and 
elementary programs (4 respondents), high school dropout rates (5 respondents), teacher 
training and teacher quality (5 respondents), equitable and flexible funding (4 
respondents), parental involvement and community support (3 respondents), leadership 
and school management (2 respondents), and school safety (2 respondents). See appendix 
A for the approach to identifying and categorizing the high priority issues. 

Curriculum and Instruction Research 

While 16 respondents indicated that research was needed to help identify and retool 
effective instruction and curricula, the emphasis differed by educational level and across 
localities. Some superintendents were concerned about elementary and middle school 
(K-5 or K-8 level) instruction, even as high schools in their districts were performing 
well in key measures (achievement, SAT scoring, and graduation rates), and wanted a 
focus on earlier schooling to prepare at-risk students for learning in high school. Other 
respondents saw high school performance declining, especially growing high school 
dropout rates that posed a major challenge in their jurisdictions. 

Using Technology Effectively 

In the area of instruction research, respondents were interested in research on technology 
in education. They wanted more research to inform them how to use technology — 
particularly the Internet — more effectively to improve classroom teaching and learning. 

They also asked for research that would help them apply technology to such areas as 
personal communication, instructional strategies, teacher quality and training, family 
problems, and poverty. They felt research could help practitioners to integrate technology 
effectively in curricula and instruction on reading, especially early- age reading. 



1 We will refer to the group as local education officials, but most of the respondents were superintendents. 
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Some respondents asked for research on effective instruction and programs to help them 
improve student achievement and school accountability. A superintendent in a large state 
with a rigorous statewide standard test saw assessment and accountability as the number 
one issue that required research: 

“We have to compete with other districts in improving student 
achievement. This is the key to accountability. We want our high school 
graduates to complete [school] with good grades and go to college. We 
need good studies to show us effective ways to do it.” 

Sustaining Improvement 

Two local education officials highlighted the importance of seeking sustained effects in 
curricular and instruction improvement. Respondents observed that effective programs at 
the elementary level would not work well in high schools: ninth graders often lacked 
basic skills and were ill prepared for advanced curriculum in secondary education. These 
respondents felt evaluation of recent innovative programs in areas known as “secondary 
literacy” and “freshman academy” would be valuable in addressing the problem. 

Respondents asked for studies to identify proven effective strategies for teachers to use in 
classroom instruction and for administrators to develop service programs. A 
superintendent expressed appreciation for the Reading First initiative: 

“Reading is indeed a very good area to focus research, resources, and 
efforts. Grants for reading instruction programs are well spent. A good 
direction, with accountability measures attached to it. I’d like to see more 
such programs in other subject areas like math and science.” 

Concerns about achievement gaps among specific student groups voiced by respondents 
were actually tied to curriculum and instruction research. Local decision makers 
interviewed wanted to have effective strategies proven by research — for developing 
either academic or service programs — to address the needs of different student groups at 
different education levels, as shown in four sections below. 

Early Childhood and Elementary Programs 

Four local education official respondents cited early childhood — including 
prekindergarten — education as a high priority for research. They saw early childhood 
education as crucial for later learning; a great deal of money had been spent on programs, 
yet it was still not clear how well they worked. These respondents needed to better 
understand what works through research that would help identify developmentally 
appropriate and educationally relevant programs. Echoing the federal initiatives, four 
respondents advocated extensive research on acquiring reading skills at an early age, 
from preschool through grade 3, as early reading was critical for virtually all other 
subjects. 

Brain Development Research 

In connection to early learning, three respondents pointed out that brain development 
research was very important and deserved strong support. Recent developments in brain 
research have shown teachers better ways to teach children by learning about the brain 
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processes. They believed further brain research could help teachers understand individual 
students’ learning styles and progressive patterns, and accordingly develop appropriate 
strategies to meet their unique needs. 

Assessment and Accountability 

Local education official respondents were divided in regard to the initiatives of 
standards-based assessment and accountability. Of the 10 who raised this issue, 
six expressed strong support, while three remained skeptical. Supporters viewed 
student achievement and assessment in core subjects as the highest priority and 
asked for extensive and in-depth studies in the area. 

Early Grade Reading Benefits 

More than half (16) of the local education official respondents agreed that the No 
Child Left Behind legislation provided a good principle for education reform and 
that the focus on early grade reading was the right approach to address 
widespread mediocre performance in elementary and secondary schools. They 
affirmed that reading was the key to learning in many academic subjects and that 
mastery of reading at an early age would benefit students tremendously in 
subsequent schooling. 

Need for Research on Standards 

Citing assessment research as a high priority, a skeptic among the local education official 
respondents asked for research to examine the very standards in relating to achievement 
gaps, arguing that the new state standards themselves need to be critically assessed, 
linking them to achievement gaps. As this superintendent put it: 

“Good standards and testing should work to help reduce the gaps, not 
widen them.” 

Two respondents were not convinced by current research evidence that standards-based 
testing actually helped improve student learning. They were not convinced about the 
means by which testing contributes to better student learning. A district head worried: 
“How do they [assessments] help? Tests take a lot of time and energy from 
teachers and students, but we don’t know how they can help learning.” 

Questions on Assessment 

One superintendent thought that research was needed to show how to “align classroom 
practice with state and national expectations of student achievement.” (This person 
referred to the No Child Left Behind Act and education reform in the respondent’s own 
state.) One question stemmed from the diverse curricula and instructional strategies in 
practice in any given district and across the country: 

“How can you teach them using different strategies in different programs, 
but expect them to perform well with a single assessment?” 

A rural superintendent did not believe a test could provide a full picture of student 
learning and questioned the wisdom of making all key decisions based on “the results of 
the one test per year.” One superintendent pointed out that because of the very small 
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number of students in his district, student achievement fluctuated dramatically from year 
to year. Assessment statistics from his district may thus misrepresent the progress and the 
instruction programs when compared with larger districts. 

Achievement Gaps 

The issue of achievement gaps, proposed by 1 1 respondents, was perceived as closely 
related to assessments and standards. It was recognized that there were wide gaps in 
achievement in students across race/ethnicity, sex, income levels, geographic areas, and 
disability status. A large urban district’s research and planning director warned: 

“We know their [underperformers] problems pretty well, but we need 
research to tell us ways to reduce gaps and improve learning. . . . Kids are 
vulnerable; they are fundamentally struggling in those urban schools and 
neighborhoods. Research is needed to inform us about what to do. We are 
at meetings and when questions are raised about what services are needed 
to help those high-risk children, people just look at each other. . . . The 
problem is worsening to a national crisis.” 

On performance gaps, one superintendent strongly disagreed with the new initiatives: 
“You cannot just say ‘No Child Left Behind’ and disregard various 
demographics and local conditions. We’ve lost a lot of funding, especially 
funding for professional development after the new policy started. We 
should integrate programs to work with all the student populations: 
minorities, immigrants, gifted and talented kids, and kids with disabilities. 

People in Washington are fooling around with ideas and know nothing 
about what’s really happening in school.” 

This respondent urged government to fund studies about how federal money had been 
spent and what impact the spending had on different groups. 

Poverty and Achievement Gaps 

Poverty, especially persistent poverty across generations and its impact on learning, was 
of grave concern to four local education official respondents. They believed deeper 
research was needed to identify strategies for closing the gap. Two respondents wanted 
more information on “methods of proven effectiveness” to improve achievement in urban 
schools for poor children. Often poverty goes hand in hand with other problems. As a 
superintendent described her district: 

“. . . 80 percent of children having free lunch and over 50 percent of 
families under the poverty level. . . . We have a high percentage of 
Hispanic immigrant children in school.” 

A superintendent working in a relatively affluent community disapproved of the federal 
government’s key initiatives: 

“We think inclusiveness should be the basis of public education. The No 
Child Left Behind law is changing that. It is excluding a lot of different 
people while saying it helps disadvantaged children. We want to see 
integration of funding for programs to serve diverse student populations.” 
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This respondent wanted research that would examine carefully the outcomes for both 
underprivileged and gifted/talented children as a result of the new funding formula under 
the law. 

Reverse Gender Gap 

A reverse gender gap concerned a superintendent who was also an active researcher: 
“We see girls are doing better than boys. It’s as if the old policy intended 
to eliminate disadvantages for females has been overdone. The research 
question is how can we work to make sure boys can catch up and reduce 
the gender gap?” 

Language Gap 

Three respondents saw learning English in bilingual education and ESL programs as 
another important issue related to research on achievement gaps. A respondent from a 
large urban district said: 

“We have a large portion of immigrant populations in [our district]. And 
this comes to the No Child Left Behind legislation. What programs and 
services do we need to effectively help these kids and to close the 
achievement gaps?” 

Geographic-related Gaps 

Geographic-related achievement gaps were another concern combined with poverty and 
other risk factors. One respondent stressed: 

“Urban areas where students and their families experience high poverty 
and high mobility are a major challenge. What kind of programs work 
effectively to address their needs?” 

Achievement Gaps Relating to Disabilities 

Two respondents also saw a necessity for more research on achievement gaps relating to 
disabilities. Students with learning, physical, and emotional disabilities were seen to lag 
behind continuously. These respondents desired more studies in special education and 
other service programs for kids with learning disabilities. A superintendent who claimed 
to see an increase of children with disabilities said he needed more information to address 
the issue. A broader issue for study was emotional and behavioral aspects of learning. 
Another respondent pointed out that relative to cognitive research, studies on children’s 
emotional and behavioral development were inadequate. 

High School Dropout Rates 

The recent rise in high school dropout rates drew renewed attention from five 
respondents, who listed the issue as a high priority for study. One observed: 

“High school completion is getting worse to the extent of a deep crisis. 

Dropout from high schools is especially bad in urban high schools. 

National statistics show only a slight increase in recent dropout rates, but 

in [my area], it is a major crisis And there is no national consensus on 

this, no agreement on the situation, or on what to do about it!” 
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Researching the Causes 

A respondent in a major urban district was alarmed by the drastic increase in high school 
dropout rates and was not sure about the cause of it. Asking for research on the issue, she 
speculated: 

“It may have something to do with the increasingly higher and higher 
standards. Students that have basic skills problems, with poor 
backgrounds, just cannot catch up with the rising standards and they just 
cannot graduate. Here we have lads who work very hard but still need two 
more years to graduate from high school. Secondary literacy is a major 
issue here. Many kids got into high school but still cannot read, cannot do 
math. They are 14, 15, 16, and 17 years old but cannot read. . . . There are 
many problems that require research to answer.” 

To illustrate how research could help address the dropout problem, a superintendent 
reported his coping strategy informed by research. Following a successful program found 
in the literature, his district started a project called “freshman academy” to target at-risk 
students in the ninth grade, with combined services and instructional programs. He said it 
had helped stabilize dropout rates in the district. 

Teacher Training and Teacher Quality 

Five respondents agreed that teacher quality and supply should be an important research 
area, but the respondents had different emphases in research priorities. One believed 
alternative teacher certification was a promising approach to address the teacher shortage; 
another doubted its value as a long-term solution. He believed research should examine 
how “fast track” teacher recruitment and training programs work, identify the effect of 
these strategies in reducing immediate shortages, and compare them with the graduate 
and undergraduate programs in teacher colleges. Generally, these respondents felt they 
needed information to identify the kinds of teaching skills required for improving student 
performance and provide a combined understanding of teaching and learning. 

Equitable and Flexible Funding 

Four local education officials cited school funding and fiscal policy research as a priority 
area. They found that many complicated, high- stakes issues in this area had not been 
adequately addressed. Two of them asked for immediate research to assess the impact of 
the federal government’s allocation of funding on local schools’ administration, 
instruction, and student performance. 

Two superintendents believed rigid control of federal funding on local operations needed 
to be examined in light of the vast changes in federal resource allocation. A respondent 
recalled that when one of his Title 1 schools substantially improved, he was not able to 
use these funds to help another school that desperately needed support. 

Three respondents were unhappy about school finance research. These respondents did 
not see many problems practically solved or even conceptually clarified. Perennial issues 
such as per pupil spending had never been clear to them, and they were especially not 
clear about how such financial measures related to student learning and performance. 
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Parental Involvement and Community Support 

The School Choice movement has exerted pressure on public school administrators and 
teachers, according to three respondents. Many were “restless with the choices,” as a 
superintendent phrased it. Another respondent perceived it as a survival struggle for 
public schools to gain parents and community support. These respondents wanted 
research to find better ways to involve parents in the system, saying that parents were 
leaving public schools for home schooling, charter schools, vouchers, and private 
schools. A superintendent urged researchers to examine a school collaborative culture 
that allows all stakeholders to work together, creating synergy through shared decision- 
making. 

Leadership and School Management 

Confronted by bewildering changes and reforms pressed by various stakeholders, two 
respondents raised a wide array of issues under the rubric of leadership research. This 
included administrator (principals and superintendents) quality, training, coaching, and 
performance measures tied to student achievement. One also called for “implementation 
studies” that showed efficient and effective ways to implement new programs. 

A superintendent was concerned about relationships with school boards and teachers’ 
unions. Under the “huge impacts of unions on what we do,” he explained, local school 
managers needed comprehensive studies to sort out constructive and productive ways to 
work with teachers, board members, and other stakeholders. He felt research on the 
impact of strikes and teacher contracts on student learning was also needed, especially in 
low-performing communities. 

One respondent noted that, in an era dedicated to small class sizes, some rural districts 
were facing serious problems as a result of low enrollment. He explained that student out- 
migration and local communities’ resistance to school consolidation were resulting in 
very small schools in rural areas. With severely limited resources, rural high school were 
offering substandard curricula and struggling with aging facilities. He felt studies were 
needed to identify effective policies and programs to cope with the problem. 

School Safety 

School safety and violence continued to be a major issue for two respondents, who called 
for research to study prevention methods as well as to assess the extent of the problem 
and costs related to it. A superintendent in a Southern district described the problem as 
acute, seriously concerning parents and staff, and including such problems as children 
being abducted, domestic violence, and illegal entry of school buildings. 

Other High Priority Issues (Question 5) 

The broad range of respondents’ research priority issues, other than those presented 
above, included: 

• A clear definition of “scientifically- based methodology,” a concept that 
dominated the No Child Left Behind legislation; 

• School schedules and double bus schedules; 
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• Year-round schools to help disadvantaged kids and improve achievement overall; 

• The impact of school crowding and ways to handle it; and 

• Alternative schools (e.g., online schools, voucher programs, charter schools, 
private schools) and evidence showing that those schools work better and are 
worth public funding. 

Use of Education Research Resources (Question 2) 

Respondents’ use of and access to research varied widely and was highly personal, 
depending largely on their organization size and resources, as well as their own 
professional networks and communication styles. 

Obtaining Research Information 

An overwhelming majority (29 out of 30) of the superintendents interviewed indicated 
that they read reports of research studies and program evaluations at least occasionally. A 
large majority (24) of respondents claimed they read research reports “most of the time” 
or “just about always.” Heads of large districts were less likely to read full-length 
research reports, largely because they had in-house staff to provide selected information 
to them. Only one respondent, a veteran superintendent, said he did not read research 
reports as he found research useless, and he made decisions largely based on his 
judgments and experience. (See appendix table D- 1 for the numbers of respondents who 
indicated different levels of frequency in reading research reports.) 

Internet and ERIC 

Virtually all respondents used the Internet as a research information source. They either 
searched the Web for relevant information, or participated in e-mail lists to receive 
information regularly. They also shared information with colleagues through e-mails. The 
superintendent of a Southern district told a story about an SAT preparation program on 
the Web that helped the district attain the average level of the state SAT test. 

ERIC, on the Internet or on CD-ROM, was mentioned by eight respondents as a source 
for research information. With its clearinghouse division of labor and specialization in 
different education subfields, these respondents felt ERIC provided reliable abstracts of 
ongoing research literature for local practitioners. The system and its databases were 
widely accessible via the Internet. Respondents also found ERIC’s literature syntheses, 
known as ERIC digests, helpful in keeping up with the fields. Two respondents found 
ERIC less useful, noting the lack of full text for all the documents it references. 

Extensive Personal Communication 

More than half of the local education officials interviewed relied heavily on personal 
communication to receive information on current developments and research in the field. 
In addition to locating information via the Internet, they used their personal professional 
networks to filter important information, dubbed “either by Internet or by happy 
coincidence” or “word-of-mouth.” They interacted closely with their colleagues, both in 
their own districts and elsewhere. One superintendent complimented the principals in his 
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district: “They are very well informed and we share research information, pretty often 
through e-mail.” 

These respondents were very people -oriented. Personal networking seemed a main 
channel for this group to obtain and share research and new program information. As 
they described during their interviews, they had the capability to and enjoyed building 
relationships with colleagues, staff, consultants, and conference speakers to access 
updated information useful for their professional work. They named these people as 
friends and valued them as personal resources. 

Professional Organizations and Conferences 

National professional associations were a strong source of information for 12 of the local 
education officials interviewed. The American Association of School Administrators 
(AASA) was the single most frequently cited information source for the respondents. The 
Association for Supervision and Curriculum Development (ASCD) was another source 
broadly used by the group (eight respondents). The American Educational Research 
Association (AERA) was also mentioned as an organization that produces research 
products and information (three respondents). Regional professional organizations were 
another important resource for five respondents. 

Nine local education officials reported attending professional conferences to share 
information with colleagues. Conferences, particularly regional conferences, were a 
valued channel for these school administrators to receive and absorb research information 
that was immediately relevant to their own districts. Two respondents tended to go to 
selected conferences to network and communicate with colleagues and researchers 
working in places similar to their own. A superintendent of a large Southern district 
regularly attended a small number of conferences and talked directly with speakers and 
implemented an improved high school schedule based on information received. 

Federal Government Sources 

Five superintendents praised the National Reading Panel’s report as a strong and 
applicable piece of research work. As a respondent pointed out: 

“Locally we don’t have the resources to do extensive research like the 
federal government has done. So it helps to see such reading research 
come out, really helping teaching and learning.” 

Fourteen respondents reported using the Department of Education’s research resources. 
Six said they had received documents from or had searched IES Web sites, and more than 
half of the respondents were aware of NCES statistical products. Two of those said that 
they appreciated the value of federal publications in offering a national perspective. 
However, at least four respondents criticized that national data and reports typically did 
not have direct application for local districts. A research and assessment director at a 
large Western urban district said: 

“NCES statistics are very important for looking at schools nationwide, but 
we found them not directly relevant to our own situations. And I don’t 
think we receive a lot of [IES] reports either, for the same reason that they 
probably are not directly helpful to our work.” 
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Journals and Magazines 

Respondents mentioned various professional journals and education- related magazines as 
a source for research and new programs, including Education Week, Phi Delta Kappa, 
Educational Leadership, Harvard Education Review, Great City Schools, and Journal of 
Staff Development. According to two respondents, Ed Week, with its electronic version, 
did a very good job in synthesizing timely information. Periodicals issued in a regional or 
state scope were also cited as good sources. 

Regional Educational Laboratories and Other Service Agencies 
Regional educational labs were cited by five respondents as helpful sources. With their 
specialties and regional locations, the labs served practitioners significantly, as reported 
by superintendents. A Regional Education Service Agency (RESA) in Georgia and the 
Board of Cooperative Education Services (BOCES), regional educational institutions in 
New York, were cited as examples of state/regional services where districts purchased 
services in curriculum development, school finance, and legal consultation. 

Internal Research and Information Services 

More than 10 local education officials described their internal communication on new 
programs and research as a regular and valuable way to keep staff up-to-date on research. 

A Northeastern local school agency held monthly District Instruction Conferences to 
share information with teachers, psychologists, and administrators, where researchers and 
writers were sometimes invited to present their work, as well. Creating a “professional 
learning community” was one of the major efforts of the district. 

Respondents from two large districts had their own research and evaluation capacity, but 
such functions were typically geared toward internal assessments, rather than exploring 
new developments from outside sources. For example, this is how a superintendent from 
the Midwest described his district’s project: 

“We designed our own assessment, in great detail, to figure out specific 
skills in reading that required more efforts, and communicated these needs 
to textbook publishers and asked them to make changes and additions.” 

The respondent agreed that local research like this is unique: “We have our own 
hypotheses and collect data to confirm them.” 

A large urban district in a Western state also ran ambitious research projects. Based on a 
systematic literature review, staff there conducted analysis of data from a 5-year 
evaluation program of 100-150 classrooms. They linked classroom observation data with 
student performance outcomes to examine key factors in instruction that affected learning 
and achievement. 

Three local education officials were very well informed about current research and 
developments. They maintained close ties to universities and research organizations, 
serving on dissertation committees or as writers/editors of professional journals. Well 
connected and active in research circles at state, regional, or national levels, such local 
policymakers might contribute a great deal of insight about setting up new research 
priorities. 
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Facilitating Use of Research 

While almost all respondents were strong critics of existing education research, 20 out of 
30 local education officials acknowledged that research helped them. Responding to the 
question on whether research had been useful, one superintendent said: 

“Absolutely. For example, the Teaching Gap (James Stigler and James 
Hiebart, 1999) summarizes videos created during the TIMSS study that 
had b rillian t observations on planning techniques in Japan. We’ve used 
this as a model toward which we’re moving gently . . . toward more 
collaborative planning. Research let us sort out the effective methods we 
ought to copy.” 

Impractical Orientation 

Many respondents (23) criticized existing research for its overly theoretical and academic 
orientation. Finding it hard to see the relevance and applicability in available research 
evidence, a superintendent argued: 

“Assessments, brain science research, and demographic studies of 
education cannot give teachers concrete ideas about teaching. Research 
should draw direct implications for practice and have direct connection to 
classroom activities.” 

A respondent who was also a faculty member at a major research university analyzed the 
situation as follows: 

“A basic problem is that researchers do not respect practitioners and 
practitioners do not respect researchers. Most researchers do not intend to 
do research to inform the practice; they have their own interests and their 
own questions. And that’s a problem! We need far more interaction 
between researchers and practitioners. Researchers must take the 
practitioners’ perspectives and raise questions from the practitioners’ 
standpoint. The federal government should facilitate the interaction. . . . 

We need ongoing interaction and collaboration between researchers and 
policymakers and teachers . . . sitting together to share information and 
thoughts in a ongoing basis.” 

Problems in Academic Research 

From some practitioners’ points of view, academic research in education is out of touch, 
self-serving, and useless in classroom teaching and program decision making. A veteran 
superintendent in a Western state summarized: 

“There may be less than one percent of the existing research that’s really 
meaningful to teachers. Much is for researchers, for getting funding, for 
career advancement, or for advocacy. ... I don’t want theories. Teachers 
need strategies, practices. Give them things that can help teaching and 
learning, things that can help kids.” 

A respondent suggested that university researchers “choose their topics after dialogue 
with practitioners in their field.” In his area, major universities should “convene a sort of 
symposium with 100 public school educators and 20 to 30 university professors and talk 
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about what matters.” There was a school- university partnership in place in Chicago, 
working to bridge academic researchers with schools, reported one respondent. 

Limits of National Data and Research 

Five respondents strongly believed education and schooling is by nature contextual and 
therefore research must run in local settings. A long-time local education official 
observed: 

“In general, national research is limited [in its value for practitioners]. You 
deal with what you are, where you are. We have unique problems that 
national research probably cannot help. National data may not be exactly 
what we want.” 

Another stated: 

“One size fits all is not right. Each state has its own problems. 

Standardization would not work with all kids. Individual kids have 
different needs, different approaches. We know Europeans and Japanese 
have their national standards, but they have very different conditions, and 
they are much smaller and less complicated systems. Research should pay 
more attention to individual needs than standards.” 

A rural district local education official said, “What works in New York doesn’t 
necessarily translate to what works well in rural southeast Idaho.” And an urban district 
planning and research director asserted: 

“. . . you cannot simply take a structure from a school where it worked to 
your own place and hope it works the same way. Mostly such things work 
only in a local setting, be it a teaching method or a school organization. 

You need to understand your own issues and the ideas underlying other 
people’s strategies.” 

A respondent felt that research sometimes came to conclusions that are seemingly 
sensible yet not practically workable: 

“Small class size is good for learning. We know that. But it requires two 
things: more classrooms and hiring more qualified teachers. We are trying 
to reduce our class size to 18 in 4 years; but we need more research to tell 
me how I can fund the effort, especially in schools with concentrated poor 
and minority students.” 

Political Influences on Research 

Three respondents expressed mistrust of (and disappointment in) existing research that 
was excessively influenced by politics. A Southwestern superintendent said: 

“A lot of statistics are just for liars. . . . Research is not for selling 
something. A lot of these days research is for selling a product.” 

Another said: 

“I’ve been in education for 35 years. Honestly, nobody really knows 
what’s going on in the area. Everything keeps changing. Today, you read 
reports about this and this, next day you read reports about just the 
opposite. There is no consistency. That’s frustrating. Education research is 
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not a science, not scientific. It is not objective. It’s politically driven. For 
example, in reading, we heard a lot about whole language learning, how 
great it was, just couple of years ago. Now we are told that it does not 
work; it’s phonic-based reading that works. I don’t know what really 
works.” 

Two other respondents argued that research operated by universities, rather than by 
government, should be the mainstay. Concerned that research conducted by universities 
had been cut in recent years, one respondent believed it was time to renew support for 
university-based research in today’s context of scientifically-based research. Another 
criticized research conducted by companies or politically- involved organizations that 
have vested economic or political interests in what they are studying: 

“Lumping things together with national statistics to express a single one 
national point of view is an example of political- driven research.” 

A superintendent called for bias- free research, saying that research must be data driven, 
rather than driven by some political agenda, and that studies should cover all aspects of 
performance, positive and negative. 

Lack of Scientific Rigor 

Eight local education officials said rigorous and in-depth studies were needed to 
understand the causes of school success. Some suggested using case studies that examine 
accountability and performance by observing in detail specific features and conditions of 
programs. Others disliked qualitative approaches, saying it was difficult to replicate the 
results, wary that without consistent and rigorous quantitative measures, accountability 
might be undermined. 

Disappointed with the poor and inconsistent quality of existing education research, a 
superintendent who actively participated in research and publishing pointed out a 
fundamental weakness of the existing education research: 

. . there are a lot of opinions, ideas, and thoughts, rather than empirical 
studies in the literature. I believe there should be more rigorous research 
with hard, quantitative data. It is hard to be sure about qualitative 
findings.” 

Variable Quality of Research 
One respondent observed: 

“A lot of good research and a lot of bad research. Some research journals 
give you really good results. Some do not. You see papers presenting 
findings without specifying procedures or limits of data collected. 

Educators are typically not good research consumers. They tend to take 
research for granted, as if all research findings are credible and can be 
used in practice. They don’t have good judgments about research 
methodology.” 
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Reserving Judgment 

Five local education officials were reluctant to make judgments on the amount and 
quality of the existing research, saying they were not in a position to do so. One 
superintendent said: 

“I’m a practitioner, not a researcher. I cannot say much about research 
quality. It goes from ridiculous to sublime. I simply use common sense to 
decide whether a piece of research makes sense.” 

Declining to assess the overall quality of the existing research, a superintendent said: 

“I tend to assume that research published is good, [that] quality issues are 
taken care of. I tend to use research with confidence. There are areas 
where research is not adequate, and more efforts are needed, of course.” 

A new superintendent explained that she had little time to keep up with research, even 
less to filter out valid pieces. She urged the federal government to support and enforce 
more rigorous peer review processes before releasing research: 

“I want to be sure that the studies I read are not isolated work and are 
substantiated by other researchers, so I can rely on them.” 

Steps the Department Can Take to Improve Education Research 

(Question 3) 

Even respondents who expressed frustration and disappointment proposed constructive 
strategies to improve the relevance and utility of education research. The group’s 
agreement underlying these recommendations contrasted with its diverse views on 
research priorities. 

Shorter is Better: Repackaging for Practitioners 

An overwhelming majority of respondents (27) recommended succinct and jargon- free 
representation of research results for local education research consumers. They 
emphasized that summaries, highlights, and abstracts were the most effective way to 
disseminate updated research information to practitioners. One said: 

‘Time is the problem, accessibility is not a problem. There is probably too 
much information out there but you just don’t have time to sort it out.” 

Another said, 

“No one can go through a report longer than 40 pages; and the best is a 
two-page five-bullet executive summary.” 

They generally felt regular dissemination of research highlights through e-mail listing or 
hardcopy publishing would work well, and suggested the Web site should provide links 
from summaries to the full-length reports for people who were interested in quickly 
locating the original texts and data. 

“Academic researchers cannot write to the general public,” a practitioner-researcher 
respondent said. She called for special efforts to transform academic research into useful 
knowledge for practitioners by thoughtfully representing the information, simplifying the 
language, perhaps even putting questions in different ways so that policymakers and 
teachers could make the connection to their work. 
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Organization and Search Abilities 

Some respondents voiced a demand for better search and indexing systems on familiar 
information services such as ERIC and government Web services that have become 
popular in recent years. Four respondents expressed dissatisfaction with the available 
search features and the many layers of grouping information in the Department’s Web 
site. They would prefer materials organized by topics or categories that were relevant and 
compatible to practitioners’ concerns and approaches. 

Believing that the information was “out there,” but not handily sorted by practitioners’ 
needs, they suggested that the Department compile various regular mailing lists to notify 
different research consumer groups (principals, assessment directors, teachers, etc.) by 
topic when new research results were released. The mailing list recipients could then read 
the listed abstracts and download the complete text as needed. They urged categorizing 
information in ways that accommodated different groups’ search habits and practical 
needs so people could quickly identify the information they needed. 

Research Synthesis and Integration 

A large number of respondents (16) complained that current research often reported 
conflicting or inconsistent findings, which was of little use and created much confusion 
and mistrust by practitioners in the field: synthesis and integration of major research 
findings was needed. A superintendent urged: 

“We need more meta-analyses. Individual studies often are not conclusive 
and have conflicting evidence sometimes. Meta-analyses allow us to have 
an overview of the findings in a particular area with integrated findings.” 

Secondary Analysis 

A superintendent argued that more resources should be allocated for secondary analysis 
and reconciliation of the existing research evidence: ‘There is more value from 
interpretation of research than original research.” He would like to see the Department do 
a good job in this and spend more resources to conduct research synthesis work. Another 
local education official suggested that the work be done by experts who were really 
f amiliar with both sides — research and practice in schools and classrooms: 

“Research needs to combine pieces of information on assessment, 
standardized curriculum, etc. into workable ideas for us to implement into 
programs. We in the field cannot put all those pieces of evidence together 
in a successful model. Researchers should do that, make information 
meaningful to teachers, not just to researchers themselves.” 

Facilitating Regional and Local Research and Information Sharing 

There was a consensus among respondents that government could not do everything. 
Regional and local studies and program evaluations were seen as key to assuring practical 
utility of research. Privately supported research, typically concentrated in local or 
regional schools, was also seen as critical for local application. 
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Local District Projects 

Local districts were conducting research evaluation projects. As described by two local 
education officials in large districts, research and evaluation projects with broad scope 
and high methodological rigor were underway. They were eager to share the results and 
hoped to get some federal support to expand and “add value” to such efforts. One of these 
respondents urged the federal government: 

“Fund more basic policy analyses at the local level. Do ongoing data 
collection at local schools. Data on dropouts, special education, reading, 
etc. Local schools don’t have resources to do such things. New support 
from government is needed to conduct ongoing information collection, to 
build local capacities for continued research at pre-K and K-12 levels.” 

Internet Distribution 

One respondent saw it as possible and desirable that 

“. . . federal agencies systematically locate and collect updated 
information from state and local policy research and program evaluations 
and distribute the information in summaries widely through the Internet. 

That way, a lot of local research information would become useful to 
many people.” 

Networking and Partnerships 

A superintendent suggested the federal government could help facilitate local and 
regional interaction and networking in sharing research information by “. . . giving 
contacts and sites across the U.S. where things are happening, so we can network.” 

There were working examples of such networking. In a large Midwest district, 
policymakers could call local research agencies for immediate information and advice in 
a local school-university partnership. The respondent from this district advocated that 
every school should have research support from credible research agencies in some form 
of research-practitioner partnership. 

Reaching Research Consumers 

Few believed that the federal government would be able to directly reach research 
consumers at local levels, so that it might be necessary to make research information 
available through national or regional professional organizations of practitioners, and/or 
state departments of education. State agencies were also seen as helpful in information 
dissemination. For example, the Texas Education Agency Web site was said to be a very 
good source for information, with a great deal of data on districts. As one respondent put 
it 

‘Tunnel through these organizations. They can do a much better job in 
identifying useful information and providing relevant stuff to members 
than the government.” 

Private Sector Approach 

The private sector’s approach to education reform was highly valued by local education 
leaders whose jurisdictions benefited from such programs. Private foundations provided 
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funding and innovative programs as models for government- sponsored research 
programs. The most prominent case was the Albertson Foundation, praised by a number 
of respondents. It provided money to schools to support innovative programs focusing on 
student learning and combining program implementation and administrator coaching with 
research evaluation. A superintendent with a 30- year career described the Albertson’s 
Creating High Performance Schools as “a breath of life,” saying that he saw “more 
change in the past 3 years [under the Albertson Foundation initiative] in schools than in 
the past 200.” Another respondent recommended that the Department’s Web site publish 
synopses of DES- funded research and nongovernment funded research, with links to the 
original sources and researchers. 

A deputy superintendent urged that the DES Web site should go beyond the role of a 
resource for federal or national research information, be used more effectively to 
disseminate locally-based research and program information, and facilitate 
communication among local programs and people across states and regions. Putting local 
content on an accessible Web site could encourage widespread sharing of potentially rich 
program information among practitioners. 

Strengthen Peer Review and Quality Monitoring 

Some respondents called on the Department to play the role of research monitor and 
reconcile results. Four respondents were wary of unreliable or erroneous research 
information, and felt vulnerable about being misled simply because they had no research 
training and often had no time or resources to filter the literature for valid findings. A 
superintendent in the West did not believe more research was needed, rather, “better, 
relevant things.” She cited the Head Start program study released years ago as very good 
research. Another respondent commented: 

“There are competing research findings. . . . There are different camps. 

There is a need for sifting through the data, . . . need someone to sort 
through it all. . . . Develop a product and we’ll buy it.” 

He proposed the federal government play this role. A third respondent suggested that a 
safeguard for local people to use valid research would be a monitoring system with 
rigorous peer review or expert review procedures, sponsored but not necessarily operated 
by the government. 

Making the Federal Bureaucracy More Responsive 

Some local education officials felt that the Department was so far removed from the 
school districts that, as one phrased it, they looked at the Department as “the people that 
pass policies you have to comply with.” One respondent resented the heavy demand for 
paper work in order to receive federal funding or to participate in federal programs and 
complained about the complicated and time-consuming paper work in proposal 
development, reporting, and compliance procedures. When asked “What can the U.S. 
Department of Education do to make education research more useful, more accessible, or 
relevant to your work?” one respondent retorted: 

“Nothing. It is not a matter of the federal government doing more; it’s a 
matter of doing less, doing less with me in paperwork, reporting. 
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meetings, and proposals. Just don’t ask that many things from local 
people. That would help.” 

Another respondent advised: 

“The federal agency needs to be less compliance [oriented] and more user- 
friendly to schools.” 

Release Data in a Timely Manner and Link Them to Programs 

One respondent observed that many federal research reports were not released in a timely 
manner, often with long delays. He said local people often needed national data to make 
immediate decisions that linked directly to districts’ programs. 

Releasing research reports together with new program announcements seemed desirable 
to local education officials. A district instruction director admitted that she would only 
search research information for specific projects, either a new federal grant or a state 
program. She suggested: 

“It would help a lot if you provide relevant research together with the 
program announcement, in the RFP. That can help us work more 
effectively. The National Reading Panel’s report is a good example of 
such research dissemination. That covers a lot of literature and provides 
useful information for teaching reading in early childhood.” 

Providing Training in the Use of Research 

According to two respondents, practitioners needed to know how to use research to make 
sound judgments about the validity and practical value of the available information. They 
recommended that the federal government provide training to practitioners on using 
existing research products effectively. A superintendent said: 

“It’s crazy to say accessibility is a big problem. But it is important to teach 
people to use research for practical purposes. How to use different sources 
for different uses at the local level; that’s still a problem.” 

In addition, respondents suggested that IES establish guidelines on research quality, 
techniques educators can use to identify and select quality research. 

Major Policy Interests (Question 4) 

Interview questions about respondents’ major policy interests generated a wide array of 
issues that may or may not have overlapped with respondents’ institutional priorities. To 
a great extent, these issues reflected local education officials’ personal concerns, ranging 
from basic research themes such as children’s self-teaching and learning styles to 
mundane school operational matters such as facilities management and classroom 
lighting. 

Flexible Funding and Decision Making 

Two local education officials were concerned about flexible funding and decentralized 
decision making. They wanted to see more studies on innovative formulas that allowed 
schools to re- allocate funding once funding was determined on the basis of student needs. 
A superintendent reasoned: 
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“Funding from the federal government relative to money from state and 
local sources needs to be studied carefully to figure out how the money is 
spent. I want to know how federal dollars are actually utilized to help local 
children and families. How do the programs supported by the money 
work? Research in this regard exists, but far from enough, and I’d like to 
see more.” 

Making Necessary Choices 

One respondent listed the following items in response to the question: facilities 
management, school size, organization of schools, education technology, online 
textbooks, and instructional resources. And then he asked: 

“How do we make choices that meet our needs? Foreign language classes 
in elementary school, maybe in second or third grade, is it feasible and 
does it make sense? Those are my personal concerns almost daily in 
work.” 

Career and Technology Instruction 

Career and technology instruction concerned another respondent: 

“Lots of money is being spent. Proponents say it really works . . . out in 
the field, it doesn’t seem to. There needs to be some attempt to reconcile 
this.” 

He would hope to have some conclusive finding on how career and technology education 
affects student life, from auto mechanics workshops to Microsoft certifications. 



IES Findings from Interviews with Education Policymakers 



Superintendents and 
Other Local Education Officials 



Synectics for Management Decisions, Inc. 



25 



CHIEF STATE SCHOOL OFFICERS 

Highest Priority Issues in Need of Further Research 

(Question 1) 

The research issues of highest priority to the nine chief state school officers (CSSOs) 
interviewed included aspects of teacher preparation and quality (six respondents), 
effective interventions in schools and in school systems (four respondents), identifying 
effective approaches to mathematics teaching and learning (three respondents), and 
enhancing reading capabilities (two respondents). 

Teacher-Related Issues 

Six of the nine CSSOs (or their designees) cited teacher-related issues as a priority issue. 
Several noted teacher shortages in their states and its potential consequences. While one 
respondent cited “professional development in relation to student performance” as the 
“more basic” of his priorities, elements of teacher qualification, training, and 
development appeared as one of the two priority issues for the majority of CSSOs. (Three 
respondents did not want to rank their two highest priorities.) The following teacher- 
oriented research issues were addressed: 

• A respondent from a state that had instituted a “multiple paths to success” 
program for teachers, as well as students, thought research on more effective ways 
to respond to the high percentage of teachers leaving the profession and the 
shortage of teachers in special areas was important. His state was exploring 
diverse teacher training and inservice professional development efforts. 

• One CSSO was interested in the “success of teachers and achievement of students 
in relation to teacher qualification, specifically teacher certification.” With his 

state experiencing difficulties in attracting competent teachers, he wanted to learn 
how current certification procedures affected teacher recruitment and 
performance. 

• Other CSSOs cited teacher quality and preparation. One noted that, because of a 
shortage of good teachers in his state, most taught four to five classes on average. 
Another wondered what would happen as the United States moves to a more 
highly qualified workforce: 

‘Will we wind up with more shortages? What will be the impact on 
cost?” 



Effective Interventions 

Research on effective interventions in schools was a high priority for four respondents. 
One was interested in seeing the outcome of money spent on programs such as tutoring in 
relation to their outcomes, which he defined as student performance. A second, whose 
research priorities focused on effective curriculum (his highest priority) and effective 
school interventions, observed: 

“Under No Child Left Behind, we are under a mandate to have all children 
at a proficiency level in reading and math in 12 years. We don’t have a 
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good handle as to what interventions are effective (outside of some 
interventions in reading).” 

Systemic Reform 

One respondent commented that his highest research priority lay in “systemic reform 
based on standards and assessments.” He called for research to show him the “validity 
and reliability of the assessment used to measure student learning and development.” 

This was critical, he believed, in order to ensure public confidence in education reform. 

One designee of a CSSO was concerned that “education evaluation be systemic.” He 
called for research to look at school districts’ structure, administration, and staff. He also 
called for more shared data both within and across states: 

“You need average states’ performance and how things are going in local 
districts.” 

Important information in determining why programs succeeded or failed were such 
factors as districts’ socioeconomic status, funding, teacher quality, and training. 

Alternatives in Learning and Achievement 

Alternative ways for student learning and achievement from pre-K through graduate 
education and beyond was a priority for one respondent. His state’s “multiple paths to 
success” program sought to identify and employ new ideas and different programs that 
would enable students to complete school successfully. He specifically wondered what 
programs and strategies were effective in middle and high schools to enable students 
from very different backgrounds to learn and graduate. 

Math Learning and Instruction 

Three CSSOs underlined research in math learning and instruction as a priority. For one, 
“professional development strategies for teachers of mathematics” was critical, closely 
linked to his concern for research on “what practices increase math capability and the 
pedagogy of mathematics.” “Best practices in math instruction” was the highest priority 
for another, and a third focused on “scientifically proven effective pedagogy, instruction, 
and curriculum programs” on research “in math learning and instruction.” Echoing the 
views of a fellow respondent, this CSSO observed that “reading is already on the national 
agenda” and that “more attention should be paid to math instruction and achievement.” 

Reading Issues 

Two CSSOs set reading- focused issues as their top research priorities. One sought 
research on best practices in reading instruction, while a second defined “early literacy, 
prekindergarten-third grade” as his top priority. He indicated that his state would be 
spending $10 milli on the first year on training reading coaches and other early- age 
reading efforts. His state “wanted to use research- based methods, and not a single 
method, to improve early reading.” 

Consideration of Standards 

One respondent felt that the whole concept of “standards” needed additional 
consideration: 
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“There isn’t a good research base that supports the logic. Is improvement 
being driven by standards, or accountability, or assessment?” 

Other High Priority Issues (Question 5) 

The CSSOs interviewed called for a number of innovative approaches to conducting 
education research, expanding the issues considered by education researchers, and 
transforming the organization of consultations on education issues. Several respondents 
also called for more research on several enduring issues in education, including the 
following: 

• Examine education issues through new conceptual lenses, such as those from 
economics; 

• Conduct “more research on nontraditional methods of teaching,” including 
smaller high schools, extending the senior year into junior college, attracting 
minorities to teach in high schools, and education for 3- to 4-year-olds. 

• Tie brain research to learning theory and bring research findings into classroom 
teaching and learning or service programs; 

• Study successful students rather than just focusing on disadvantaged students and 
students-at-risk, and take a “clearinghouse approach” to sharing information on 
successful students and schools; 

• Hold more meetings in the regions, with fewer meetings in Washington, DC; 
widely sharing research information collected at the local level; 

• Identify the best approaches to special education and ESL students; 

• Involve parents, especially parents with low reading levels; 

• Research effective instruction through the Internet and through live video and 
compare the difference in student performance; and 

• Study student performance and accountability, in programs combing instructional 
improvement with data collection on student performance. 

Use of Education Research Resources (Question 2) 

One of the CSSOs interviewed reported reading research studies or reports of program 
evaluations “all of the time,” and three reported reading these materials “most of the 
time.” One respondent directly stated that he consulted research findings “only some of 
the time ” Three CSSOs’ responses to this question, however, were ambiguous: 

“I’ve covered a lot of bases.” 

“I do what I can. We have a series of deputies and directors working on 
different areas using research in their work.” 

“My staff scans research literature for me on a weekly basis. I read 
summaries and abstracts. It’s part of my work to be familiar with what’s 
going on in the research front.” 

It would probably be fair to say that these respondents consulted education- related 
research at least “some of the time.” 

One CSSO flatly stated that he “never, or hardly ever” read research reports, adding “I 
don’t want to bury myself in research reports.” He did modify his statement somewhat by 
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saying that he might read something if it was “short.” Later in the interview, however, 
this CSSO named specific publications he consulted. (See appendix table D- 1 for the 
numbers of respondents who indicated different levels of frequency in reading research 
reports.) 

Obtaining Research Information 

Internet, ERIC, Regional Labs 

Four of the CSSOs interviewed indicated they used the Internet and ERIC for research 
information. In addition to ERIC, one respondent specifically cited the Department of 
Education, and one mentioned “other clearinghouses.” Three named regional educational 
laboratories as research sources. Two respondents specifically mentioned Mid-continent 
Research for Education and Learning (McREL), and one respondent spoke of the North 
Central Regional Educational Laboratory. One long-time CSSO remarked: 

“Regional labs are great sources, McREL in particular. I want [IES] to 
know that when [it’s] making changes, keep in mind that the regional labs 
and ERIC work pretty well for teachers and local policy people.” 



Staff 

Four CSSOs cited staff as research information sources — these individuals ranged from a 
research assistant “solely devoted to research and information services” to subject matter 
experts. 

Publications 

Education Week was identified by four CSSOs as a source of information. One reader 
used the publication as a “good start” for checking major issues; if he found something of 
interest, he went to the full report and related literature. One CSSO cited the publication 
Education Next and the newsletter Gadfly as research resources while another respondent 
cited “trade magazines” as a resource, without further specification. 

Organizations and Colleagues 

The Education Commission of the States and the Association for Supervision and 
Curriculum Development were organizations named by single individuals as research 
resources. Colleagues and fellow CSSOs were a source of research information for 
another. 

One CSSO, who relied primarily on research summaries prepared by his staff, indicated 
what he did not do. He had “no time” to use the Internet. He did not often attend 
professional conferences as “many conferences are politically orchestrated” and not much 
value for learning new practices and programs. 

Facilitating Use of Research 

Effective Research Tools 

Five of the nine respondents described the kinds of tools that would help them make 
more effective use of the enormous volume of education research. Two identified efforts 
to enhance communication among important education stakeholders as crucial and one 
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called for increased dissemination through recognized and respected national associations 
of education policymakers. Specific suggestions included: 

• Set up and maintain a Web page where people can look up everything on key 
issues; 

• Provide a clearinghouse on key issues/research findings; 

• Develop a comprehensive listing of research — a “Research Hotline” — released on 
an ongoing basis, highlighting all recent work in a short, simple manner, 

• Provide executive summaries, condensed versions of research available to CSSOs 
and their staffs; and 

• Organize categorical information with good search features to locate the right 
information quickly. 

Clearinghouse of Meta-analyses 

One respondent said it would be helpful to have a clearinghouse of meta- analyses to 
“assess research claims of the value of particular evaluations — an assessment of 
assessments.” With so much information available, and an increasing number of studies 
not put through peer review processes, this respondent said “[the reader] doesn’t know 
the validity” of research work or how to assess it. 

Bringing Policymakers and Practitioners Closer 

“Close communication with practitioners and policy people” was viewed by one CSSO as 
fundamental to encouraging more regular use of research. He noted his collaboration with 
American Indian educators in studying issues of Native American education and the 
needs of Native American children. Another pointed to the “widening distance” between 
the federal government and school districts and between state government and local 
schools. He called for “work on this disconnection” between schools and government to 
create a “climate where research is valued and understood by practitioners,” noting that 
“teachers and principals are not well trained to follow up and understand research.” He 
stated that research is “on the bottom of the agenda in most districts.” 

One of the interviewed CSSOs felt that the Council of Chief State School Officers and 
the Education Commission of the States would be valuable dissemination vehicles for 
education research information. Research findings could be e-mailed to these 
organizations and then circulated among their members. Regional educational 
laboratories, universities, as well as the Department, could also disseminate research 
findings. 

Steps the Department Can Take to Improve Education Research 

(Question 3) 

“Aggregating research and resources” from its unique national perspective was a function 
a number of CSSOs saw as a particularly valuable role for the Department. One CSSO 
articulated the thoughts of several others: 

“A lot of research that needs to be done would be impossible for a state to 
do. The Department is uniquely positioned to do strong, effective 
research.” 
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A second important role suggested by respondents was giving a voice at the table to 
education practitioners, listening to their concerns as they presented them, and 
responding to these concerns. 

Department as Information Broker 

The CSSOs who were interviewed had a number of imaginative suggestions for 
enhancing the role of the Department as a pivotal information broker. 

Department Providing Perspective 

Concerned about topics in the field of education that “tend to have a very narrow focus,” 
as well as a somewhat “incestuous” research community, one respondent saw a role for 
the Department in taking “a perspective that is not so wrapped up” in a particular topic. 

He identified the issues of gifted education, school choice, or alternative certification as 
cases in point, observing that there were “vested interests” in some of these subjects in 
colleges of education. He felt a valuable contribution by the Department would be “to 
look at different ways of looking at things and developing a research base.” 

Clearinghouse on Effective Practices 
One CSSO suggested: 

“The Department could create a clearinghouse and disseminate 
information to the states [and] share use of effective practices.” 

Speaking of his own state, he observed: 

“We don’t do a good job of knowing where a particular practice is used 
and its effectiveness. Further, we don’t know what is going on in other 
states.” 

Another respondent noted that he and his fellow CSSOs “all shared the concern” that 
there be more sharing of information on best practices and strategies among states. 

Partnerships for Research 

One respondent suggested the Department establish partnerships with states and 
nonprofits to conduct research. He noted that the U.S. Department of Health and Human 
Services had successfully implemented partnerships with organizations such as the 
Manpower Development and Research Center, which provided technical support to both 
the federal government and the states. He also called on the Department to “get people 
from 10 states together to talk about what’s going on in each state in assessment and 
research.” 

Another respondent called on the Department to “provide information on what the 
schools are doing and tie it to things we are required to do by federal programs.” 

Streamlining Research Reports 

Respondents commented that research reports needed to be easy to read and understand. 
As CSSOs and their staffs need to access information outside of their offices quickly — 
literally anywhere — research reports cannot be too long and too complicated. One 
respondent suggested the Department could “help such a process.” 
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Clearinghouse Services 

Two CSSOs focused their remarks on clearinghouse services the Department could 
provide. One respondent called for either creating a clearinghouse on educational 
research directly through the Department or funding its creation somewhere else, such as 
the Council of Chief State School Officers, the Education Commission of the States, or 
the regional educational labs. 

Providing Key Findings 

One respondent specified several issues where he thought the Department could provide 
“key findings”: dropouts, teacher recruitment, early childhood education, leadership 
training (how schools of education are preparing future leaders of K-12 education), and 
teacher preparation — encompassing such issues as nontraditional programs and 
alternative certification programs. 

Importance of Helping End Users 

Two CSSOs focused on the importance of communication as a critical element in 
fostering exchange of ideas and practices between the Department and the states. One 
observed the need to concentrate attention on the needs of “end users” and focus on 
questions from a practitioners’ perspective. He called on the Department to “create an 
environment and mechanism to help end users,” and to find ways to make them pay 
attention to research and to use research information. Another strongly underscored the 
need for the Department to have “close communication” with all groups, policymakers, 
teachers, researchers, and government agencies. “Let me and my staff participate in your 
meetings. . . . directly talk with you in your office,” he asked. “Proximity and presence 
are important,” he observed. Continuing, he lauded the work of the regional educational 
labs and centers (“these people really know the local schools and help”) and was 
concerned that a national Department of Education body “may not be able to replace” 
these institutions. 



Major Policy Interests (Question 4) 

While the policy interests identified by respondents reflected the diverse education issues 
facing these CSSOs, issues of education quality (four respondents) and education 
financing (two respondents) were the most frequently noted. Two respondents noted 
other issues, and two took the opportunity to reiterate points they made earlier. 

Education Quality 

Policy interests focusing on educational quality touched on a variety of concerns. One 
CSSO formulated his policy interest at the broadest level: creating a more effective 
public education system. He went on to say that 21 st century education “has more market- 
like characteristics and incentive structures.” He thought that “90 percent of any quality 
problem is an issue of the system, not the people in it.” Another CSSO’s interest focused 
on “closing the achievement gap” among racial/ethnic groups and “raising achievement 
levels for all kids.” A third CSSO was interested in improving student learning, the 
impact of early childhood opportunity on later success, and learning of “success stories,” 
as well as issues of leadership and teacher quality. 
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Turning his attention to the quality of postsecondary education, one CSSO’s policy 
interest was “how to ensure that all students are receiving an education at the level of 
quality of Tier I universities.” He was concerned with preparing his state’s students for a 
postcollege future in the workforce and wanted students who did not have access to Tier I 
universities to “be as competitive as they can be” with Tier I university students. 

Finance 

One respondent wondered how the federal government would redefine a funding role 
consistent with the programs under the new Elementary and Secondary Education Act 
(ESEA). State issues of adequacy and equity in funding were of interest to him, as well. 
Another respondent was concerned with the consequences of financial stringency on state 
education activities and wanted to know “how states are doing more with less, what 
legislatures are doing, and what is going on at the national level.” 

Research 

Non-English-speaking Students 

One CSSO was concerned with “English learners.” Noting that a large percentage of the 
students in his state came from non- English- speaking backgrounds, he remarked that this 
area was a “prime” one for research on such questions as the appropriate language to use 
in educating these students, how using English or students’ native languages would affect 
the learning of different student groups and how to assess student performance. “English- 
only versus bilingual education” was a critical issue in his state. 

Comprehensive Research Resources 

One respondent reiterated his concern with finding easy ways to access literature and 
research and called for “one stop shopping where you get comprehensive research 
resources.” He observed that ERIC did not have the full text of all the documents it 
referenced and that ERIC users often had to pay to get the full text of certain documents. 
Another CSSO responded to the policy interest question by restating his support for 
regional educational laboratories and centers. 

Usefulness of Research 

Five of the nine respondents affirmed the usefulness of research in addressing their areas 
of policy interest or providing guidance. One observed, “I’d be in trouble without 
research information about schools and students’ needs.” Another, who consulted 
research findings only sparingly, remarked that research had helped him in “working 
through” his own ideas. A third respondent remarked that he had recently gone through 
research on early childhood reading and found that “there are many good ideas that can 
be applied in our programs.” He also saw brain research as “another very valuable 
research area” giving “good answers to our questions.” 

Commenting on the quality of research on his major policy interests, one CSSO stated 
that “on a scale of 10 for quality,” he would rank the research “between eight and nine.” 
Another respondent felt despite the criticism of education research, “there are good 
research studies” and praised the research-based, very practical, and effective work of 
McREL. 
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Gaps in Research 

Several CSSOs pointed out gaps in education research, with a particular focus on the 
applicability of research findings to education practice. Two specifically pointed to a 
dearth of research on rural education. One other concern, emerging here as in other points 
in the interviews, was the need for attentiveness to the education research needs of states 
and localities. 

Addressing the gap between research and implementation, one respondent stated: 

“The problem for education research is that it is not well related to 
practice. Many good ideas are not implemented in classrooms. We need 
training for administrators and teachers to catch, understand, and use 
research findings.” 

Another respondent noted that he was “a strong believer in research” and tried hard to 
help his state agency gain and use research effectively. He noted: 

“Politics always gets involved in education research, which makes it hard 
to use the results. . . . We need to help people understand and value 
research. People may have different issues and concerns and may want to 
measure different outcomes. You can do that, but with some core 
outcomes measured for comparison.” 

New Approaches to Education Research 

One CSSO felt that education research was “not really” useful to him and found that 
research results were infrequently translated into what-to-do kind of information. He 
found very little information out there that could make things happen. He commented on 
the “fairly quick” spread among physicians of new drugs, practices, and equipment as the 
result of efforts by a variety of institutions and organizations to help implement these 
innovations. He believed a similar approach was needed for education research, whereby 
teachers were helped to implement new ideas and practices so that they could “quickly 
put research into use to improve teaching and learning.” 

Another CSSO indicated: 

“[We] need a strong, factual, empirical basis to support policy approaches. 
Education [now] goes on what our gut tells us.” 

Regarding the usefulness of research for his policy interests, one respondent stated an 
outright, “No, if results mean anything.” He also commented: 

“If you have agreement near the boundaries of the discussion, then you 
can reach a consensus. This is mostly an area of political difference.” 

He felt that research could be looked at cynically. Elaborating, he stated that a lot of 
policy was being driven by research that was being driven by a client. “There is a 
political bias built into the structure,” he concluded. 

Getting Research Up to Standards 

One CSSO respondent commented, “Education research deserves being called lousy.” 
Comparing this research to that in health and human services, he found education 
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research to be inferior and researchers’ training inadequate. There was “a long way to go” 
in getting education research up to the standards he sought with “strong methodology and 
scientific rigor.” He called for “more random design and well controlled experimental 
studies.” He saw too much anecdotal research and sought objectivity and neutrality in 
research, as well as more searching research questions: 

“Ask more why questions to deepen research.” 

Another CSSO cited “a lack of research, especially in reading and math instruction” and 
felt there “should be more research readily available, easy to understand, applicable to the 
classroom.” This respondent was concerned, as well, over the quality of some of the 
scientifically-based programs that the Department had highlighted. 

Rural Concerns 

Addressing a perceived shortage of rurally- focused research, one CSSO felt that most 
education research “is designed for urban areas with little consideration of Native 
Americans” and was more focused on Blacks and Asians. Native American children, this 
respondent noted, are the lowest performing subgroup in Western states. Research was 
also lacking in the area of educational technology use in small, rural areas — for example, 
what are the best distance-learning approaches for rural areas? This respondent also saw 
an urban slant in textbook design, observing that “instructional materials don’t have 
rivers and mountains and cows” — the daily reality of children in rural areas. A second 
CSSO observed that “rural education is rarely studied” and asked if there was a “national 
agenda to study and improve rural education?” 

Research at State/Local Levels 

Two respondents highlighted a recurring state concern for increased attention to research 
needs at the state and local levels. One CSSO, focusing on the activities of the 
Department, observed that when programs were earmarked and research funded “not 
much is communicated and agreed upon with state and local people.” Another respondent 
commented: 

“From a state point of view, we need to communicate and compare with 
other states on many issues in public education.” 

Examples of areas in which he sought comparative data were in assessment, 
accountability, security, facility building, and leadership. He also called for more 
research in professional training, performance-based budgeting, and appraisal of 
employees and programs. 
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STATE HIGHER EDUCATION EXECUTIVE OFFICERS 

Highest Priority Issues in Need of Further Research 

(Question 1) 

Of the 10 state higher education executive officers (SHEEOs) interviewed, half of them 
cited issues focused on student retention in the higher education system. Tied to this were 
questions of student achievement and the performance gap among students of different 
backgrounds (two respondents). Another significant area of attention lay in the financing 
of higher education (four respondents). Their concerns lay in the impact of increased 
tuition costs on lower income students. One SHEEO was specially interested in strategies 
that states and the federal government could develop in assuring higher education 
affordability. 

Three SHEEOs addressed the issue of developing indicators or measures of the 
performance of institutions of higher education. The changing nature of the 
postsecondary student body underlay the research concerns of many of the SHEEOs 
regarding student retention. This was directly referenced by three SHEEOs in their calls 
for research on nontraditional students and distance education programs, research on 
diversifying postsecondary education faculty, and research on different pathways to enter 
teaching careers at the K-12 level. 

For a number of respondents, their two highest priority research areas were directly 
linked and they declined to designate one issue as ranking higher than another. For those 
who were willing to rank their priorities, fiscal issues and student retention/student 
success had equal numbers of adherents, three each. 

Making Use of Research 

Several SHEEOs expressed their research concerns at a broad conceptual level. One 
called for “research on research.” He observed that research studies indeed exist on “what 
makes for effective schools,” but this information had not been used by policymakers: 
“Why doesn’t all this research get used to determine policy?” 

He also sought research in the broad area of performance indicators. Directing this 
interest “at any level of education,” he wanted studies on the development of 
performance indicators, how they were being used, how they worked, and “determining 
where the most effective performance indicators are.” 

Increasing Success Rates 
One SHEEO asked: 

“What do we need to do to greatly increase the rate of success in all levels 
of education for all our people?” 

He noted that his state was concerned with closing the gap in higher education with 
attention focused on issues of participation, success, quality, and research. 

Another SHEEO respondent, whose research priority lay in “improving student 
achievement and how to get there,” also wondered: 
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“Why hasn’t educational reform produced increased student 
achievement?” 

He and his state colleagues were concerned with the connections between student reading 
success in early grades and increasing the number of high school graduates, and getting 
people into the workforce and through some form of postsecondary education. Another 
sought “measures of quality in higher education.” He called for research to identify more 
and better measures of the quality of learning. For legislators in his state, “traditional” 
indicators such as employment after graduation, income, and job status were 
unsatisfactory measures. 

Student Retention Research 

One SHEEO asked, “How effective is our student retention program?” He wanted “more 
research” to tell him ‘how to improve student retention” and how current instructional 
models could be improved to do that. A second respondent, concerned with retention and 
completion issues, felt that “student- level information” was badly needed and that 
aggregated data were no longer sufficient. She commented that IES had institution- level 
data and the states had individual- level data, so collaboration between states and the 
federal level was critical: 

“Only with individual- level data can we learn how students change, move, 
complete, or dropout from colleges.” 

New Approaches in Higher Education 

Another respondent was concerned with closing the performance gap in higher education 
among diverse racial/ethnic groups and income levels: 

“We see kids come into colleges with similar levels of performance, but 4 
years later, some graduate, and some do not.” 

He wanted to see research on “how we can help low- income and minority students 
achieve at a high level and complete college.” 

Demographic Research 

Changes in the demography of student populations determined the research priorities of 
two SHEEOs. Observing the “drastic increase of a new ethnic population” in his state and 
nationwide, the first felt that “a key to helping minority kids enroll and complete higher 
education is to have diverse faculty in our institutions.” Since legal problems could result 
from such a strategy, he advocated studies to indicate “some feasible approach” to the 
issue. 

The second SHEEO noted that the number of “nontraditional students” was “rapidly 
expanding” and their needs and approach toward education were often “different from 
those of traditional college students.” She felt the “old model doesn’t fit” and observed 
that many of the nontraditional students were taking noncredit courses at the 
postsecondary level and preferred distance learning. She called for national data on 
nontraditional students and distance education programs: 

“We need new knowledge about this population and new ways to serve 
them with distance learning programs.” 
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Nontraditional Pathways to Teaching 

Nontraditional pathways to teaching were the focus of another respondent. He noted the 
“very serious shortage of teachers” in his state and sought new strategies to attract 
students into school teaching careers: 

“We are interested in seeing what research can say about nontraditional 
pathways.” 

Retention and Completion Research 

One SHEEO called for research on the reliability of the student completion rate formula 
for higher education and cohort tracking for completion. Another was concerned with 
retention in both community colleges and in 4- year colleges. Retention was linked for 
him, as well, to the issue of student transfers from community colleges to 4- year 
institutions: 

“It appears that a large number of community college attendees are not 
prepared to do college work and do not have the ability to do it.” 

He believed “there must be something else that is needed,” as his state spent a lot of time 
and energy enabling students to transfer from community colleges to 4- year schools. 

Financial Issues 

Focusing on fiscal issues in education, one SHEEO’s research priority lay in determining: 
“Are we funding things appropriately; are we getting the most for the 
buck?” 

Another asked: 

“What are really reliable fiscal indicators of school health, well-being, and 
cost-effective uses of resources? There is really no one place that gives 
you a sense of what the most salient fiscal indicators are.” 

Her state had looked at data from the National Association of College and Business 
Officers, but found nothing definitive there. 

Affordability and Access to Higher Education 

The question of affordability and access to higher education was a concern of another 
SHEEO. He noted that his state’s budget had declined by more than 10 percent, but the 
number of people seeking a college education had increased. He observed that people 
coming from outside his state were an important source of professional skills: 

“We are not doing very well in providing college education to people in 
the state.” 

A fellow SHEEO cited the concern by many in the SHEEO community regarding the 
affordability of higher education. Their discussions focused on defining the roles that 
states and the federal government could assume in this area. There was no “consensus as 
to what to do about the problem.” He stated that with the current rates of increase in 
college costs: 

“We are stripping low- income fa mili es of the ability to send their kids to 
college.” 

He called for studies on strategies for state and federal collaboration on this issue. 
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Questioning the effectiveness of higher education financial aid programs in helping 
needy students complete higher education, one SHEEO commented: 

“We know we need specific goals and a continuing monitoring 
mechanism.” 

He observed that student debt default is very high and many students would never 
graduate from college once they defaulted on a student loan: 

“We need accurate information and effective ways to address the 
problem.” 



Other High Priority Issues (Question 5) 

In addition to elaborating on their original research priority areas, several SHEEOs set 
out a number of new issues: 

• How students and their families perceive affordability, select a particular college, 
and make payments; 

• Immigrant attitudes toward and understanding of higher education; 

• The world of work from the perspective of student preparation to that of broader 
state and national economic development; 

• Vocational education; 

• The state of science and technology education; 

• The success of high school assessment tests and the SAT/ACT in predicting 
college success; 

• The cultural role that education plays from birth to fourth grade; and 

• Character education. 

One respondent expressed a general query: “Are we doing all we can be doing to 
improve educational capability?” 

Finally, one respondent asked for “more surveys, like this one, to get peoples’ 
input when you make changes.” 

Use of Education Research Resources (Question 2) 

All of the SHEEOs in the sample read research studies or reports of evaluations of the 
programs in which they had an interest. Three of the respondents indicated that they read 
such studies or reports “only some of the time,” two said that they read these materials 
“most of the time,” and three indicated that they “just about always” read studies or 
reports. The two remaining respondents indicated that they read summaries of research 
reports prepared by staff — one emphasized that he did a “selective” reading of these 
summaries, particularly in his areas of interest; the second did not indicate how 
frequently he read staff-prepared summaries. (See appendix table D- 1 for the numbers of 
respondents who indicated different levels of frequency in reading research reports.) 

Obtaining Research Information 

The Internet was clearly a valuable tool for these education policymakers, with five 
SHEEOs specifically mentioning the Web as a research resource. The specific research 
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resource most frequently mentioned was The Chronicle of Higher Education. Six of the 
10 respondents mentioned the Chronicle by name. 

With reference to Department research resources, four SHEEOs cited NCES as a 
resource, with two specifically mentioning IPEDS. ERIC was mentioned by four 
respondents, but it was generally rather far down the fist of sources consulted. 

Organizations Cited 

Regional or national organizations focusing on higher education issues were an important 
source of research information for a number of SHEEOs: 

• The Western Interstate Commission for Higher Education (WICHE) (three 
respondents); 

• Southern Regional Education Board (SREB) (two respondents); 

• SHEEOs’ own national association (two respondents); 

• Education Commission of the States (two respondents); 

• National Governors Association (one respondent); 

• College Board (one respondent); and 

• American Association of Community Colleges (one respondent). 

An education think tank, the National Center for Public Policy and Higher Education, and 
a foundation, the Miliken Foundation, were also cited. 

Facilitating Use of Research 

Good Web Site Design 

Underlining the importance of the Internet, SHEEOs cited the importance of good Web 
site design more frequently than any other feature that would make it easier for them to 
use research information on a regular basis. Web design issues noted included: 

• Ease of use; 

• User-friendly index systems; 

• Information easily accessed by topical areas; and 

• A comprehensive clearinghouse providing links to educational material, indexed 
by subject matter. 

Comprehensive Clearinghouse 

Describing the Department Web site as “sometimes difficult to navigate,” one SHEEO 
called for “a centralized information Web site set up by topical areas that would be easier 
to navigate.” This was echoed by a fellow SHEEO who asked for “a comprehensive 
clearinghouse that maintains links to relevant educational material, indexed by subject 
matter.” 

Another respondent spoke of the “luxury” it would be to “assign crackeijack 
professionals” to nothing but identifying research that was credible and relevant. 

Up-to-date Information 

Two SHEEO respondents called for more up-to-date information — one of the 
respondents specifically calling for an updated, simplified EPEDS. Another observed, 
“Education research must be constantly updated.” He felt that “all topics in education” — 
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specifically mentioning early childhood education, reading, and postsecondaiy 
completion — need data collection on a regular basis: “You don’t want dated and sporadic 
information.” 

Commenting on the difficulty of retrieving tuition studies across states from the 
Department Web site, one SHEEO observed that the Chronicle of Higher Education did a 
better job of using current data than did IES. “We are in a world where things change 
rapidly,” he said, and the 4- year- old data provided by the Department were not very 
helpful. 

Access to Original Data or Full Texts of Research Reports 

Access to original data or access to the full text of a research report was of importance to 

two respondents. One SHEEO stated: 

“Research is always available, but data are not. It is important to have 
access to public data. We want to do our own analysis.” 

Another called for “more national data” to supplement state data: 

“We want to know where we are and where we should do more and spend 
more, relative to other states and the nation. We now have piecemeal data 
on institutions, we need to integrate this data with the national data.” 

Need for Summaries and Short Reports 

One SHEEO commented, “Executive summaries are wonderful.” Size is critical, he 
noted, calling for short reports and 2- to 3-page summaries. “I just cannot do lots of 
reading in my office,” he said. For him, in-depth reading comes only at home and is 
linked to his programs and interests. 

Steps the Department Can Take to Improve Education Research 

(Question 3) 

Wider Array, Broader Range of Data 

Providing a wider array of data and presenting a broader range of issues are steps seven 
of the 10 SHEEO respondents thought the Department could take to make education 
research more useful, accessible, or relevant. Three of this group called for more state- 
level data: they particularly needed comparative state data on such issues as graduation 
rates, tuition fees, programs, and faculty. There was interest, as well, in multi-year data. 

Others called for information on the following subjects: inclusion of data within IPEDS 
on private occupational schools; best practices in performance outputs; and providing a 
clearinghouse for sites that have research capabilities. One SHEEO asked that “data be 
made widely available for all institutions and higher education agencies.” 

Design issues concerning the Department’s Web site were raised by two SHEEOs who 
wanted such features as a good user-friendly index, clear tabs, and buttons. Others called 
for more updated information on the Department Web site and links to original sources 
from summaries of research reports. Another voiced concern over the politicization of 
educational research and indicated that he would “like to see more work reviewed and 
monitored by external committees.” 
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Information on Private Occupational Schools 

One SHEEO related her call for EPEDS data on private occupational schools to the 
increase in the numbers of nontraditional students, who tended to go to these schools 
(e.g., schools providing training in information technology). States needed data that 
“accurately describe nontraditional students and the processes they go through in 
institutions in comparison to traditional college students.” She called for research on 
standards, consistent definition of programs, educational levels, and costs at the private 
occupational schools. Performance and accountability measures were needed, as well, “to 
see if our state follows or diverges from the national trend in operation and outcomes.” 

State-specific Data 

A respondent called for “easier access to state- specific data,” recommending that the 
Department Web site be “more aligned to specific topics rather than the hodge podge it 
is.” Another called for the wide availability of data for all institutions and higher 
education agencies. His state needed to “validate data” and do its own analysis on 
different issues: “We want to look at the changes across years and states.” 

Facilitating Research Projects 

One SHEEO felt things had “gotten worse” in the last 2 years and that the Office of 
Management and Budget was a “big hassle” to go through for research projects, badly 
slowing down important work. He felt that OMB had “over- controlled research 
programs” and that they were “playing politics .” He called for external committees to 
assure good research, as committee members were “more objective and able to give a 
rational assessment” of the research before them. 

Major Policy Interests (Question 4) 

Financing Higher Education 

Four SHEEOs indicated their major policy interests concerned varied aspects of 
financing of higher education — including issues of financial modeling and the worth and 
costs of college education for both government and families; financing of higher 
education linked to performance measures; and the demand elasticity of charges to 
students and enrollment completion in higher education. 

Performance Measurement 

Performance measurement was a policy interest for two SHEEOs: one was interested in 
“taking a fresh look at our institutional studies, measures of performance, and operation,” 
while the other asked, “How do we know that what we are doing has a positive impact on 
the state?” 

Link between Education and the World of Work 

One SHEEO expressed concern over whether his state was “producing people for the 
jobs we need.” The link between education and the world of work was a policy focus 
noted by two more SHEEOs — one of whom identified his interest as “seeing people well 
prepared to grow from level to level to a job.” 
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Other Areas 

Other areas of policy focus identified by individual SHEEOs included: 

• Effective programs for helping students with physical or learning disabilities to 
learn and graduate from college; 

• The transition from 2- to 4-year colleges; 

• Developing a public agenda for education and seeing that it was acted upon; 

• The amount of education necessary for both individuals and society to move 
forward; 

• Lack of compliance with public policy; and 

• Academic freedom. 

In addressing the question of research usefulness, availability, and quality in relation to 
their major policy interests, SHEEOs cited bias in research, concerns regarding research 
quality, gaps in research, the applicability of research to practice, and real world 
implementation. 

Quality of Research 

With reference to research quality, this group raised a number of issues. One SHEEO, 
observing that the quality of some research reports was “pretty bad,” proposed that 
“research should be juried — reviewed extensively before release.” He felt that external 
review panels could be very helpful in assuring quality and monitoring research 
operations. Others were concerned about the reliability of research findings and their 
scientific value. One observed that a lot of research could be found in the area of 
educational improvement: 

“. . . but is it relevant . . . was the design of it any good whatsoever? 

Because someone says we have a control group doesn’t mean that the 
research is good.” 

Noting “inconsistent” research evidence on key education issues, one SHEEO called for 
“integrative” research that combined and reconciled existing research evidence and 
indicated that this was now more important than research on specific topics. Another 
observed: 

“The best way to improve the quality of the information is to use it.” 

Gaps in Education Research 

Among the gaps in education research noted by the SHEEOs were studies on indicators 
that work in tracing the impact of education on economic development. They also saw 
the need for research on “true value added outcomes” of higher education — the degree of 
knowledge gained by students upon completion of college as compared to their 
knowledge level upon entering college. Other areas where SHEEOs identified lack of 
research were in the transition from 2- to 4-year colleges, the skill sets that an individual 
needs to succeed in the workplace, and what business requires in graduates to ensure their 
success. One SHEEO called for more sharing of research information across states and 
noted that the federal government could be helpful in this area. 
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Research Bias 

Three SHEEOs spoke to the issue of research bias: 

“There are always group interests and political agendas behind the issues, 
but we need to reduce partisanship in research. Even studies conducted by 
professional education organizations have bias.” 

‘You need to know the background of the studies, the organizations that 
supported the studies, and the researchers so that you can be aware of 
possible ideological bias or special interests.” 

“Education becomes the political football for anyone running for governor 
or senator and gets jerked around every 2 years.” 
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STATE LEGISLATORS 

Highest Priority Issues In Need of Further Research 

(Question 1) 

The two highest priority areas of concern of the two state legislators interviewed were 
financing K-12 public education and the connection between teacher training and student 
achievement. 

“There is simply not enough money” to pay for the demands placed on state and local 
education agencies by the No Child Left Behind legislation, one legislator stated. “You 
need to do a study of the actual costs” of implementing this legislation. 

The connection between a teacher’s education — both the initial degree and continuing 
education — and improvement in student academic improvement was the focus of the 
second state legislator’s attention: 

“Does National Board Certification improve classroom instruction?” 

The connection between class size and learning, particularly for students from poor or 
minority families, was also of concern to one legislator who observed: 

“Some kind of conclusive report by the federal government may be 
needed to settle this issue.” 

Other High Priority Issues (Question 5) 

Financing school programs and their connection with educational outcomes was again the 
focus of the two state legislators. Their issues included: 

• Produce federal templates, guidelines, or benchmarks that linked to program 
performance or budgets and funding to help states determine the effectiveness of 
state programs to help kids, particularly “poor kids,” learn; 

• Study the financial connections between education expenditures and the academic 
results sought; and 

• Measure the impact of teacher salary increases on student achievement. 

Use of Education Research Resources (Question 2) 

One state legislator read research studies or reports “only some of the time” and the other 
read them “fairly often.” (See appendix table D- 1 for the numbers of respondents who 
indicated different levels of frequency in reading research reports.) 

Obtaining Research Information 

Both legislators used research resources available to them through their involvement in 
the work of state legislatures. One indicated that the legislative library of the state 
legislature was a source of research information. The other cited the research help 
provided by the research staff of the National Conference of State Legislatures as a 
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valuable resource. Other resources used by the legislators included: attending conferences 
(this opportunity was diminishing with state budget cuts), Internet search capabilities, and 
Internet forums. One legislator noted the help of a research assistant in obtaining research 
information. The second legislator discussed the initiative taken by a school of education 
within the state’s public university system to compile and distribute a brief monthly 
overview on education research issues to education policymakers. This was particularly 
valuable as it was “relevant.” 

Facilitating Use of Research 

For both legislators, information overload was an ongoing problem. Both sought 
information that could be easily accessed. One observed, “Any research document with 
more than five bullets is hard to go through.” This respondent added that there was “no 
way” for legislators to read through documents with more than 40 pages. The second 
legislator did not want “to be bombarded” with research reports and called for “more of a 
selective process,” with synopses of information. 

Steps the Department Can Take to Improve Education Research 

(Question 3) 

Both legislators called for easily accessible brief summaries of education research. In 
addition, one legislator called for “no bias” in the reports made available. 

Major Policy Interests (Question 4) 

One respondent’s interest lay in how to have the state’s “severely underfunded” budget 
meet the minimum quality standard set out by the state itself. The second legislator’s 
policy interests focused on teacher performance and incentives to bring out higher quality 
performance. This legislator specifically called for improving teaching performance so 
that good curriculum and high standards would be taught to children, as well as studying 
the effect of different compensation systems for teachers (systems linked to teachers’ 
performance in the classroom). A final concern was “trying to make sure that public 
education is focusing on educating children and not so much on [teachers’] self interest. 

One legislator expressed concern with political bias and personal interest in “much of 
education research,” and that even Department reports were sometimes biased by the 
political agenda of the administration then in power. 
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GOVERNORS’ EDUCATION POLICY ADVISORS 

Highest Priority Issues in Need of Further Research 

(Question 1) 

All four of the governors’ education policy advisors (GEPAs) interviewed cited issues 
relating to teacher quality as a high priority (and the highest priority for two respondents). 
Accountability and assessment concerns were of highest priority to one respondent and a 
shared priority for another, with a third advisor designating “closing the achievement 
gap” as her top priority. Three of the respondents also cited issues of student evaluation 
and assessment as high priorities. 

Teacher Issues 

For one GEPA, studying “the knowledge and skill of the teacher” and assessing its effect 
on teacher performance was a critical research question. She noted that there were “such 
[high] expectations” for teachers, but insufficient research on the subject. A second noted 
there was “ a lot of conflict” in the data regarding teacher quality. Another called on the 
federal government to study what he saw as “the failure of higher education,” both to 
“attract and train talented young people to become our teachers” and to “provide 
sufficiently well trained teachers.” This GEPA was also interested in determining the 
difference made on student learning by spending more money in such areas as “staffing, 
technology, and data collection.” A fourth GEPA said her state focused on teacher quality 
issues, with significant activity in the area of teacher training. She observed that it was 
not enough just to recruit new teachers: “You need to dig deeper.” 

Testing and Accountability 

Referencing the requirements of the No Child Left Behind legislation for assessment of 
student progress, two GEPAs sought research on testing and accountability. One GEPA 
asked: 

“Would the existing tests really help; do they really help improve student 
achievement? There is no evidence that those tests actually help inform 
teachers and policymaking and then help improve learning and teaching.” 

Another respondent observed: 

“What kinds of evaluation of students can be indicative of their progress?” 

Similarly, respondents wanted to know what reading programs “really work” for 
prekindergarten through third grade: 

“There are many reading programs available in the market, but confusion 
over what really works.” 

For one advisor, “finding effective approaches to closing the achievement gap between 
White and Black students” was the highest research priority. She asked: 

“What are effective approaches used within schools and state policies that 
are effective in closing this gap? Putting a high quality teacher [in the 
classroom] would close the gap.” 
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Appropriate Curricula 

One GEPA was interested in the alignment of the curricula of middle school, high school, 
and postsecondary education; 

“We’re starting to find middle school students aren’t getting appropriate 
curriculum for high school — even students in a college prep context — to 
go into postsecondary education.” 

Other High Priority Issues (Question 5) 

Two of the respondents identified issues in professional development and school 
leadership as additional high priority areas: 

• Public misunderstanding of the concept of teacher professional development; 

• Specific needs of different student groups in professional training and 
professional development programs; and 

• Preparation of school leaders and school leadership issues. 

One respondent, echoing earlier observations by fellow advisors, noted: 

“There is research to show that if you do certain things, students will learn 
and schools will improve. Why is it that schools don’t improve when we 
have all this information? What is the disconnect?” 

Uses of Education Research Resources (Question 2) 

The GEPAs interviewed were frequent readers of education research: three read research 
studies or reports on education issues “just about always,” and the fourth consulted 
education research reports “at least three times a week.” One advisor commented, “I 
always try to look at any research.” Another observed that he looked at education 
research studies “. . . daily, though I don’t have enough time. I’m very selective. I know 
what deserves my time and what does not.” (See appendix table D- 1 for the numbers of 
respondents who indicated different levels of frequency in reading research reports.) 

Obtaining Research Information 

The GEPA respondents shared many of the same strategies for obtaining research 
information, but with some individual differences related to their professional 
backgrounds and the conditions in their states. All of the advisors made use of the 
Internet, and most also consulted professional journals. ERIC was cited by one advisor, 
and another mentioned his “research staff’ as a source of information. Two respondents 
cited Education Week, and one cited the education- focused Phi Delta Kappan. Journals in 
management and business education were consulted by a GEPA with a background in 
business and management, who also turned to Business Week, the Wall Street Journal, 
and electronic mailing lists for research information. 

National and Regional Associations 

Research information sources included national and regional associations with specific 
focus on education issues: 

• Southern Regional Education Board (one respondent); 
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• Education Commission of the States (one respondent); and 

• Education Trust (one respondent). 

GEPAs also cited research- related assistance from colleagues in state government offices 
and elsewhere. Conferences were a particularly useful resource for one advisor, but 
conference costs had led this person to use teleconferences increasingly. 

One respondent gained considerable research assistance through involvement in the 
state’s special commission focusing on education issues. The commission had been 
looking at education research studies to see their applicability to the state’s concerns and 
had invited out-of-state education researchers to address the commission. Further, this 
state had established a relationship with Harvard University researchers on literacy 
issues. 

Facilitating Use of Research 

One advisor with access to rich information on education research stated, “We’re 
comfortable. There is plenty of research out there we can go to.” Another expressed 
satisfaction with the What Works Clearinghouse role in identifying and selecting 
valuable information for teachers and administrators. He observed, “We need more 
information about teacher quality,” based not only on teacher training and academic 
credentials, but also on “more background and assessment information.” 

Time-saving Initiatives 

Two other advisors called for time-saving initiatives enabling them to access education 
research quickly and easily. One advisor called for having research available “online, and 
not having to go through a million gyrations to get it.” This respondent found ERIC time- 
consuming, and the research information provided by the National Governors 
Association and the Education Commission of the States was “material available in the 
Chronicle of Higher Education and newspapers.” This person wanted “something 
available online that is research-based.” 

The second respondent called for summaries of research, condensing, for example, the 
findings of a 200-page research report into one page: 

“Make it simple, clear, and short. If you give people more than 40 pages, 
no one will read it. Perhaps the Education Secretary should release a 
Friday Letter every week for practitioners, local policymakers, and 
administrators.” 

Such a publication should “use simple language” to tell people what was studied and 
what was found. 

Steps the Department Can Take to Improve Education Research 

(Question 3) 

The GEPA respondents had a number of suggestions for the Department. One advisor 
who advocated concise summaries of education research called on the Department to 
“send such research summaries to everyone” — including teachers and parents — through 
e-mail. These short summaries, he argued, “must make practical sense” so that they could 
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be understood and used. A second called for “a Web site with specific key issues, topical 
areas, and information on where one can go for information on a specific topic such as 
closing the achievement gap . . . where one can go to access quality research.” 

A third advisor expressed frustration with conflicting research findings: 

“This week we are told this worked; next week we are told it did not. It 
confuses people, confuses teachers, and cannot help improve student 
achievement.” 

The fourth was concerned that the research put forward by the Department “is research, 
reliable and credible and not just the report of a study. ... [It is] very important that the 
Department not put a spin on the research.” 

Major Policy Interests (Question 4) 

The major policy interests of the respondents ranged from early childhood education 
through higher education. One GEPA whose responsibilities spanned the full spectrum 
was concerned with the “effectiveness of programs, what kinds of policies can we put in 
place to effect change?” A particular “passion” was “staff development” — how to better 
help people out in the field. 

Another GEPA focused on early age learning opportunities for children, not just in the 
advisor’s own state, but across the nation. Two other respondents were interested in the 
linkages among different components of the education system. One was interested in 
seeing “how you take data sources to create a comprehensive data system that can be 
used to follow students to the point where they get jobs.” This advisor wanted to “look at 
students on a longitudinal basis . . . what is happening?” The second advisor, who termed 
himself “a higher education person,” sought to “try to understand how the failure in 
higher education contributes to the problems in K-12.” 

Three of the four GEPA respondents found the education research they were using to be 
useful in providing guidance. One respondent was quite satisfied with the quality of 
research found through identifying “credible sources.” One respondent, after answering 
“yes” to the question, noted that it was always necessary “to dig around” to get the right 
information. 

Amount and Quality of Research 

Another GEPA felt that there was “just not enough” education research in this 
respondent’s area of interest, workforce development. Business literature in this area was 
more focused and “has more meat on it” than the education literature. This respondent 
observed that there was “a lot of meddling” in the education research literature and that 
there was “so much squishy research in education.” The reasons for this might have to do 
“with the nature of the client,” in that, “We get very scared when we think of messing 
with children and what they learn.” Three GEPAs had specific critiques of the amount 
and quality of educational research, underlining such issues as conflicted or biased 
research findings, methodological directions in research, and bandwagon effects. 
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Problems in Existing Research 

One respondent found “problems in the existing research” and used the research on 
charter schools as a case in point, observing that every school had something to say and 
that evidence supporting or opposing charter schools was not clear: 

“People have different interests in it and are biased in making judgment. 
Research should be conducted to clarify the issue.” 

A second respondent thought there was “too much research, not too little” and felt that 
quantitative research, especially, had been “overdone.” This person called for more 
“implementary research” — research that “encourages people to use and implement 
established research ideas in their practice.” This respondent proposed giving grants to 
people to implement programs in schools. 

A third respondent felt: 

“We latch on to gurus in education, far more quickly than elsewhere. We 
go with the name and tout them all over the education media and don’t 
deeply explore their concepts. We have touted experts who don’t have 
expertise.” 
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CONGRESSIONAL STAFF MEMBERS 

Highest Priority Issues in Need of Further Research 

(Question 1) 

Two top priorities of the six Congressional staff members interviewed were improving 
student achievement and teacher quality. Three staff members listed improving 
achievement for all students as their highest education research priority. Staffers 
specifically highlighted the need to improve student achievement in math, reading, and 
science, and one staffer called for a knowledge base of methods and materials that would 
enable America’s educators to improve the outcome of schooling for all students. 
Research in the broad area of teacher preparation/training and teacher quality was the 
highest priority for three other Congressional committee staff. Two respondents focused 
on teacher education that would enable graduates to teach effectively amid education 
reform by offering the latest research in reading and math. 

One Congressional staffer listed several other priority areas. These included early 
childhood education, which the respondent saw as the critical stage that would influence 
children’s future learning and success in the workforce. His other priorities were special 
education and adult literacy practices to enable adults to function “in everyday life,” 
access to higher education, and English- language learning. 

Other High Priority Issues (Question 5) 

Only four Congressional staffers responded to this question, the others indicating that 
their earlier remarks adequately defined their interests. Their issues included: 

• Older students and dropouts in middle school and high school, especially in 
relation to high school functioning; 

• English-language learning about non-native speakers — particularly what is 
effective in non- Hispanic learning; 

• “What we know and don’t know about teaching and learning”; 

• Research quality; and 

• Cross-disciplinary syntheses of education research focused on their use in 
education policy and practice. 

Use of Education Research Resources (Question 2) 

Five of the six Congressional staff members read research studies or program evaluation 
reports either “most of the time” or “just about always,” with only one staffer consulting 
such resources “only some of the time.” (See appendix table D- 1 for the numbers of 
respondents who indicated different levels of frequency in reading research reports.) 

Obtaining Research Information 

Congressional staffers often received research information sent directly to the offices of 
their House or Senate committee or their Congressional member. They received reports 
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from what one staffer termed ‘'public-private institutions,” such as the Urban Institute and 
the Aspen Institute. Colleagues and journals, such as those published by the American 
Educational Research Association and Phi Delta Kappa were resources, as were think 
tanks and universities, including associations of universities such as the American 
Council on Education. Two staffers specifically cited the Department (one specifically 
mentioning the Department’s Planning and Evaluation Service) as a research resource, 
although one staffer indicated that he “hardly ever” used ERIC as it was “very 
cumbersome” and “you never get what you want.” This staffer distinguished between the 
needs of academics and professional researchers and people who “work on the Hill.” He 
thought ERIC would be most useful for someone “in the field.” Only one staff member 
specifically mentioned the Congressional Research Service as a research resource. 

Facilitating Use of Research 

Three respondents were satisfied with their ability to use research information, noting 
that “it’s pretty easy to have the information needed” or that the information “wasn’t hard 
to use.” 

Two staff members called for a database — each calling for somewhat different capacities. 

One staffer looked for an easily accessible database containing research summaries; the 
other, knowing that databases already existed, suggested “an improved online database of 
research reports, indicating the quality and scientifically- valid uses of such reports.” One 
staffer, who already accessed a wide array of research sources, wanted to find out “about 
other sources” — getting away from her usual research sources. 

Steps the Department Can Take to Improve Education Research 

(Question 3) 

Summaries and syntheses of information were important to these Congressional staff 
respondents, enabling them to have a good overview of important research findings 
without having to sift through a great deal of information. Four of the six respondents 
specifically called for the Department to provide this service. The nature of the 
summaries called for varied somewhat, including the following elements: 

• Synthesizing a large body of work in one topic; 

• Dis tillin g education research by indicating the top five research projects and the 
top five conclusions; 

• Inducing the “best, most disinterested cognitive scientists to synthesize what is 
dependably known about learning” in a form that can have direct practical 
application to schools, and making it available to the public in an easily accessible 
Internet database; and 

• Developing research syntheses that “work out” the contradictory findings in the 
research literature and providing more background information to enhance 
readers’ understanding. 

Several of the staff interviewed were particularly concerned with ensuring that research 
findings were applied and used. There was interest, as well, in having the Department 
facilitate the flow of relevant research findings to Congressional staff, with one staffer 
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calling on the Department to “convene Hill staff to get them up to speed on a topic on 
which there is significant legislative activity.” 

One staffer paid particular attention to the Department’s structure and funding of 
educational research. She suggested that the Department put a higher priority on 
education research and “spend energy and money attracting the best researchers around, 
creating a higher profile for [IES].” She further called for the Department to “develop a 
long-term research agenda such as that of the National Institutes of Health” and work on 
“long-term research projects that are truly meaningful.” 

Major Policy Interests (Question 4) 

For a number of respondents, their major policy interests were those of the committee for 
which they worked or the legislator who employed them. Hence, higher education was a 
policy interest of three respondents and job training/workforce development was a policy 
interest cited by two staffers. Teacher quality was a policy interest also cited by two 
staffers. Research focusing on “why poor kids can’t get what they need” was a staffer 
concern, as was English- language learning. One respondent defined encouraging rigorous 
review of education research as his major policy interest: 

‘There needs to be an invigorated agency that is capable of carrying out a 
coordinated, focused agenda of high quality research, statistics, and 
evaluation, with as many random assignment experiments within the 
constraints of practicality and funding [as possible].” 

Impact of Politics 

Two staffers specifically spoke of the impact of politics on the use of education research. 
One staffer indicated that “to make a strong case, I need more than one body doing 
research: issues tend to be politicized.” Another talked of “popcorn research — you stick it 
in the microwave and it’s ready. A lot of popcorn research is used by both parties.” He 
further defined popcorn research as the research that is often served up by advocates for 
particular positions. 

Defining Best Teaching Practices 

One staffer, citing E. D. Hirsch’s book. The Schools We Need and Why We Don ’t Have 
Them, stated that sufficient research now existed, so a strong consensus definition of best 
teaching practices in every critical field could be made. The staffer cited a number of 
these best practices agreed upon by a number of national associations and observed: 

“The findings of research emphatically do not accord with the reforms 
currently being recommended by the education community. These 
practices are widely used in virtually every public school in America, all 
without valid, evidence- based research to back them up.” 

Another observed that “random experiments need to be done in math instruction where 
there is a paucity of concrete evidence on best practices,” comparing this to the 
significant volume of evidence on reading instruction. 
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Quality of Research 

Nearly all of the Congressional staff members interviewed found the education research 
they were using to be of value, with several calling for more research. “A lot of research 
needs to be done to develop policy,” noted one staffer. Another stated, “You have to 
filter” the research, and it is “not always a good match” for the questions asked. Another 
staffer said there appeared to be less research on higher education than on K-12 issues, 
but “what research there is regarding higher education seems to be better than the 
research regarding K-12.” One staffer commented that finding a particularly innovative, 
imaginative research approach in one research study would inform how she viewed and 
approached the results of other studies. 
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EDUCATION ASSOCIATION EXECUTIVE DIRECTORS 

Highest Priority Issues in Need of Further Research 

(Question 1) 

The education association executive directors in the sample represented organizations 
whose concerns encompassed a broad range of education interests, from early childhood 
through adult and continuing education. Most respondents focused on research that would 
address their associations’ specific interests and needs. 

Student achievement and teacher recruitment, retention, and quality were high priority 
issues for five of the 10 respondents. The areas of student assessment and early childhood 
education were high priority research concerns for four people. Additional research 
priority areas included broad school system change, accessibility of alternatives to public 
schooling, higher education accountability, new directions in higher education, cognitive 
research, and limited- English proficiency. 

Student Achievement Tests 

Areas of research interest in the broad area of student achievement included the impact of 
poverty on achievement, furthering student achievement in reading, math, and science, 
and reducing the student achievement gap. ‘Teople know nothing about the implications 
of poverty on performance,” one respondent said. “Some states do a good job on this,” 
another noted, “but we need to know more about this.” 

Teacher Recruitment and Retention 

Teacher recruitment and retention was a concern at both the early childhood and K- 1 2 
levels. One respondent stated, “There is a prolonged shortage of good teachers in both 
public and independent schools.” She pointed to shortages of teachers in technology, 
foreign languages, and in rural areas and observed that the image of teachers and their 
rewards, as well as the nature of school financing, caused teaching to be a less attractive 
career choice. Another respondent called for understanding and developing a solution to 
an annual turnover rate of some 50 percent of the people who teach young children. A 
third respondent indicated that “the jury is still out” on the question of content versus 
pedagogy in education. His organization’s membership “would say that teacher quality is 
a huge issue.” 

Student Assessment 

On student assessment, one respondent would like to see research that would provide 
guidance in “gauging adequate yearly progress in students.” She wanted to know specific 
aspects of instruction that resulted in optimal progress by students: findings that could be 
used to intervene in the process. Another respondent’s interests focused on the impact of 
testing on student performance and on dropout rates and both the positive and negative 
long-term effects of testing. 
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Early Childhood Education 

Early childhood education in the context of school readiness was another key research 
interest. A crucial research question in this area was “how to define quality in a caring 
and learning environment for young children.” One respondent observed that “quality is 
more than curriculum” and the world of education “is community- based and quasi- 
experimental.” He argued: 

“[The] education setting is a broad-based system and not a laboratory 
bench. In the need for experimental rigor, we are forgetting that children 
and communities are not hermetically sealed.” 

School System/District Performance 

“Strategies that improve the academic performance of big city school systems” was the 
highest priority issue for one respondent. He observed that there was a fair amount of 
research on what it took to turn around individual schools, but there was no research at 
the district or school systems level on what it took to turn around an entire school district 
or system, particularly one with a high proportion of poor kids: “There is a huge research 
void.” Continuing, he observed that there was almost nothing that said which school 
districts were doing a really good job, and indicated the difference between them and 
school districts doing a poor job. 

Independent Schools 

One respondent sought research which would inform her association’s membership on 
how to maintain independent curricula and manage school financing so that independent 
schools were accessible to diverse groups of children and promoted their academic 
success in these environments. 

Higher Education 

Two executive directors focused their research priorities specifically on issues in higher 
education. The first respondent addressed the issue of accountability in higher education. 
He felt that more sophisticated performance measures were needed for all higher 
education institutions and “studying and establishing standards for accountability” was 
key: 

“It was important to show students, communities, and states how their 
investments are used, the value of the service and the return.” 

A second area of research interest for him was in demonstrating the value of higher 
education to tax payers. There was a real need for indicators showing how undergraduate 
education performed and produced; no such measures were now available. 

The second executive director looked for more research focused on distance education, 
especially for underserved and nontraditional students: 

“It is a big challenge to meet nontraditional students’ needs, which are 
different from those of the conventional college student.” 

Internationalization of education was a second priority area for this respondent: 

“[This phenomenon] requires foreign language acquisition, global learning 
and study abroad. Today’s global economy and work market demand new 
strategies and new programs.” 
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Basic Cognitive Research 

Basic cognitive research was a high priority for another respondent. She was interested in 
cognitive research related to children’s learning in math and science and relevant to 
curriculum and instructional development. Research in this area could help teachers 
understand kids’ learning processes and cognitive growth. Another director identified 
“better information about how to teach limited English proficiency students in bilingual 
or English as a Second Language programs” as an important research priority. 

Other High Priority Issues (Question 5) 

Other priorities proposed by executive officers included the following issues: 

• Create an IES vision of what’s best for kids, and see how people judge and 
contribute to school success; 

• Closely communicate with the public and local communities and disseminate IES 
products through workshops and meetings; 

• Identify “what excellent teaching looks like”; 

• Investigate the difference individual teachers make in school outcomes; 

• Examine the transformation of the role of the headmaster in independent schools; 

• Study the link between early childhood programs and later student achievement; 

• Identify “what kids need to know and will be able to do when they graduate from 
high school”; 

• Study reading comprehension among high school students; 

• Understand “how kids learn and how they test”; 

• Collect longitudinal data to address the student transfer issue at the postsecondary 
level; 

• Standardize measures and definitions in tracking students; 

• Re-examine definition of performance levels in NAEP; 

• Continue experimental design-based studies but do not neglect other research 
approaches; 

• Avoid bias in education research (two respondents). 

Another director recalled that the Academy of Education had been serving as an advisory 
body to IES for a long time. This institution had given a great deal of effort and thought 
to education research and she did not want to see this relationship with IES “scratched.” 

Use of Education Research Resources (Question 2) 

Of the 10 association executive directors (or their designees), three respondents read 
research reports studies or program evaluation reports “just about always,” with four 
consulting such reports “most of the time,” and three respondents “some of the time.” 

(See appendix table D- 1 for the numbers of respondents who indicated different levels of 
frequency in reading research reports.) 
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Obtaining Research Information 

Half of the association executives indicated that professional journals were an important 
research resource for them, with one association executive specifically citing the 
materials produced by the Association for Supervision and Curriculum Development and 
the Consortium for Policy Research in Education. Education Week was cited by one 
respondent in this group. The Internet was a research source noted by four respondents, 
while ERIC was specifically mentioned by three. The Department was referenced by 
three respondents — one whose association worked with the Department requesting 
specific information on its area of special interest; one who looked at contractors’ reports 
to the Department; and a third who cited NCES as a research resource. 

Three of the directors interviewed identified their associations’ staff members as research 
sources. Two respondents cited their own in-house research as a resource, one observing 
that his association published its own research journals. Regional educational laboratories 
were mentioned by two respondents, one specifically identifying the Northwest Regional 
Educational Laboratory. Colleagues and “sundry relationships with the research 
community” were resources, as well, for three respondents. Universities, think tanks, 
books, meeting proceedings, and materials sent to associations were all further research 
resources for these respondents. 

One respondent obtained research information both from attending conferences of other 
organizations and working to organize her own association’s national conference to share 
research and program information. Other approaches included consulting reports from 
foundations and philanthropies, and serving on advisory committees. 

Facilitating Use of Research 

The activities and services of the Department were the focus of most of the responses to 
this question, with six directors commenting on Department activities. Five of these 
discussed the Department’s Web-based services, calling for better Web page design, 
enhanced usability features, and more timely research information, with three 
respondents specifically calling for improvements to the NCES Web site, one focusing on 
ERIC, and another respondent addressing the IES Web site. 

Improving Web Services 

Many directors supported Internet-based information services. Commenting specifically 
on IES, one respondent stated that it needed to improve its Web service, “providing better 
indexing, more complete and detailed categorization, and more useful search functions.” 

He called for “more highlights and summaries of current research — perhaps released like 
a newsletter every month.” Another executive director specifically suggested that NCES, 
which she thought had a lot of information on its Web site, “improve the index and 
search features.” 

One executive director liked to go to ERIC, but found it difficult to acquire full-length 
documents through the system. She observed that ERIC was only as good as the people 
sending it information. What would improve ERIC, she felt, would be enabling it to have 
“one-stop shopping for research information.” 
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An intensive user of the Internet observed that many government agencies failed to 
classify their data, so he could not quickly locate material relevant to his work. “NCES 
has plenty of statistics, but needs to organize them well.” He called for government 
agencies to contact people to understand what they want and, on the basis of the 
responses, improve online services. 

Improving IES Services 

One executive director felt IES should do much more reviewing of research in its 
publications. Another respondent, while praising NCES as providing “good survey data, 
statistics, and analyses,” felt that “[IES] did not do a good job.” She observed: 

“I’m not blaming [IES], itself. [IES] is a political football. It has very poor 
resources and staff, [and] is very much politicized and highly unstable.” 

Summaries vs. Full-text Reports 

While some directors welcomed electronic summaries of research reports — one indicated 
that his association’s members would read nothing but executive summaries — other 
executives indicated their interest in working with the full texts of research reports and 
making their own distillations of findings. One respondent stated that he hated to “have 
research being funneled by an administration”; another stated that he wanted to “delve 
into [a report] from start to finish.” Several respondents suggested that executive 
summaries of research reports be directly linked to the full text of the research reports. 

Releasing More Timely Data 

One respondent thought the federal government’s data were not timely, and were delayed 
for years. Citing the example of the NCES 1999-2000 Schools and Staffing Survey, he 
noted that the data were just released in 2002, a 2- year delay, which reduced the use of 
the data for decision-making. 

Effect of “ Marketing ” Research 

Concerned with the marketing culture affecting the dissemination of research material, 
one respondent called for “a fine definition of a well-researched study.” She observed 
that universities and associations were linked, in one way or another, to organizations 
which marketed their materials. She underlined that she was not talking about deliberate 
bias, just that marketing enveloped the whole process. She felt IES could provide a 
“uniform code of instructional review.” Everyone else, she said, had a stake in marketing: 

“None of us is exempt.” 

Steps the Department Can Take to Improve Education Research 

(Question 3) 

The directors who were interviewed raised broad concerns regarding the nature of the 
research supported by the Department. Three respondents called on the Department to 
ask more basic, philosophical questions in framing its research agenda, set forth quality 
guidelines, and support basic scientific research. Three other respondents called for less 
esoteric research, with research studies more directly linked to policymaking. Several 
made concrete suggestions on areas for additional research, such as providing more state- 
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and local- level data and providing more finely disaggregated data on a variety of 
subjects. 

Take Educators ’ Views in Research 

One respondent called upon the Department to “look at things from an external 
viewpoint,” taking the educators’ position in framing research. He advised asking 
“fundamental questions” such as “Why should we improve adult literacy?” rather than 
posing only technical issues “such as the best way to improve adult literacy.” A second 
respondent believed that some research might not be initially useful or initially relevant 
as basic scientific research was not meant to be useful to immediate practice: 

“. . . that kind of zeal about usefulness and relevance is actually making 
research political. . . . Usefulness and relevance all depend on people. 

What is useful for researchers is not necessarily useful for teachers. 

Different people have different needs.” 

She felt the Department had “no understanding of what’s really out there in schools and 
classrooms.” 

One director called on the Department to make its research “more relevant, seeding and 
nurturing high quality research on young children.” He felt there was a bias in the 
Department as to what constituted quality in research and a lack of agreement as to what 
was high quality research. Calling for more flexibility and understanding of different 
points of view, he underscored the need for openness: 

“When there is rigidity and dogmatism in embracing different models, you 
limit yourself; your peripheral vision is shut off.” 

Understand User Needs 

Calling for a more pragmatic research perspective, one director observed there were 
studies that were too abstract, too general, and had limited value for policymaking. To 
link research to decision making, he suggested the Department do more surveys and 
focus groups with information users to understand their needs. Another executive saw the 
existing research as “too arcane and too narrow.” This respondent added, “Nothing in the 
Department at all informs the practice of big city schools.” There were program 
evaluations and more academically-based studies, but little was helpful to practitioners. 

Provide State- and Local-level Data 

One respondent remarked that “state data are always more useful than national data” and 
asked for more data to be made available at the state level. She observed that more local 
data were needed to compare schools districts with each other and with a national picture. 
She asked IES to explore coverage of local district data in its surveys and to release this 
data “as far as local agencies are willing.” Other specific suggestions included a call for 
more specific research information by demographics, school systems, sectors, and types, 
and respondents mentioned that more data were needed on private school students, 
schools, programs, and performance. 
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Major Policy Interests (Question 4) 

While these respondents articulated a broad range of policy interests, some issues were 
highlighted by several respondents: enhancing student achievement (three respondents), 
financing of schools and of early childhood education (three respondents), leadership 
development (two respondents), nurturing at K-12 and university levels (two 
respondents), school choice (two respondents), and access to education (both to 
independent schools and to postsecondary education — two respondents). Additional 
policy foci are indicated below. These respondents also had comments on the kinds of 
research they needed or found useful. 

Specific Policy Interests 

Student Achievement and Professional Leadership Issues 
Addressing the issue of student achievement, one respondent described his interest as 
“what can drive” urban school systems forward in student achievement. Professional 
leadership issues were of concern to two respondents, one focusing on “what makes 
effective principals [and] superintendents,” and the second “interested in how to provide 
opportunities for professional leadership training and growth in higher education.” He 
was also concerned with how to build capacity in higher education and make it accessible 
to various groups. Other directors were concerned with policymaking related to 
achievement and reducing the student achievement gap. 

Education Financing 

Education financing was of particular importance to one respondent who queried: 

“How do you create a high quality system that is also affordable for 
parents. How do you cost out quality?” 

He observed that the average cost of preschool was greater than attending a public 
university: 

“You sacrifice quality when parents need childcare. If they don’t have 
childcare, they can’t work.” 

Role of Principals 

One respondent was particularly interested in the role of principals: 

“[There are] holes in our knowledge. There are many issues [unanswered] 
relating to the role of today’s principals. What are the features of effective 
principals? How to select and attract quality principals, especially in poor 
performing schools? What should be covered in professional development 
for school administrators?” 

This respondent added that research was needed to clarify the principal’s leadership role 
in relation to student performance and classroom instruction. She stated that another 
important dimension where further research was needed was clarification of the 
principal’s leadership role in relation to student performance, instructional leadership, 
and professional training vis-a-vis that of excellent and veteran teachers in the school. 
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Assessing Student Gains 

One respondent found research useful to her in both “positive and negative ways.” She 
was interested in seeing how studies were done and particularly wanted to learn about 
student gain — to make a judgment on what was effective in promoting student 
achievement. She felt that this was a “critical time” for assessing individual student gains: 
“We haven’t had data to make definitive conclusions regarding student 
gains on an annual basis . . .states haven’t had the data until recently.” 

She felt that existing data was based on laws of averages and was “a waste to look at.” 
What was important to her was to “get down to individual student data.” She commented: 
“We’ve had a number of meetings; we need to get state superintendents to 
make the data [on individual student gain] available . . . along with data 
that correlates to individual teachers.” 

Privacy considerations, she noted, would hinder the release of such information. 

“ Lost Curriculum” Issues 

One association executive director’s interest lay in “the lost curriculum” which she 
defined as “the impact of standards- based curriculum on arts and foreign language 
programs, primarily; but also on social studies.” With so much emphasis on math and 
reading, she observed, “kids are not getting a well-rounded education.” She strongly 
believed that kids should graduate with “a rich education.” 

Other Interests 

School reform, voucher programs, and charter schools were interesting to several 
respondents, with particular concerns about ensuring the institutional independence of 
independent schools and providing a model of diverse educational approaches for other 
school systems. 

Many other policy interests were also mentioned, including: effective instruction and 
teacher quality, governance of schools, standards, and assessment; education for poor 
children; raising U.S. citizens’ basic education levels; and providing opportunities for 
adults to acquire basic skills. Other concerns were special education for children with 
disabilities and the needs of limited English proficiency in children. 

Kinds of Research Needed or Found Useful 

More Specific Information 

Respondents called for more detailed, updated information and much finer specificity of 
research findings for the subjects and groups of interest to them. Several noted areas 
where research studies did not seem to exist or described gaps in existing research, others 
indicated new models and perspectives in addressing education issues, and one 
underscored the need for assuring the integrity of education research. 

“The more specific information, the better for our use. Information should 
be provided for specific groups, purposes and schools.” 

“Sometimes you see studies that are too generic, or that do not directly 
address practical issues and policies. . . . For example, research in student 
persistence is pretty good . . . but college transfer and student mobility is 
not well understood. Students change programs, institutions, and states. 
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Little has been understood about the reasons, the processes, and the 
impacts of such changes.” 

“When [education research] fails, that’s because there’s not enough 
comparative information for me to sort out differences. We need data for 
more population subgroups, data at different education levels, 2-year 
programs, undergraduate, graduate, and workforce training and 
performance. . . . Honest research and data are essential. That’s the most 
important thing for government research. The integrity of research is 
fundamental to federally- supported research. I hope [IES] can keep its 
research that way.” 

Relevant Research 

Underlining the centrality of research to her work, one respondent observed, “I cannot 
function without research . . . but that doesn’t mean all research is good. There are good 
projects, for example, the Early Childhood Study conducted by NCES.” She felt “case 
studies are not very useful to policymaking” and believed that “most researchers have no 
idea about how schools work.” She thought education researchers did not understand the 
basics of school operation, school financing, how to measure per pupil expenditure, and 
student-teacher ratios. 

Research Outside of the Field of Education 

Another association executive did not find the research out of IES to be useful to him. He 
felt that “education research will have to get very creative” in order to address such issues 
as funding. He found some good research from foundations such as Pew, the Carnegie 
Foundation, the Packard Foundation, and the Foundation for Child Development. The 
quality of research, he felt “is very varied, a mixed bag” with disagreements over what 
was quality in research. 

Two directors found research outside of the field of education highly useful. One cited 
research by the military and corporations, where he located information on issues of 
institution innovation and changes applicable to school system change, the other had used 
more business model research than education research on organizational and management 
issues. Citing the Baldridge quality criteria and Total Quality Management (TQM) as 
important tools, the latter asserted that business/management research based on results 
and on “what works/what doesn’f ’ was applicable to education. 
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APPENDIX A 
METHODOLOGY 

The target population was policymakers working in various jurisdictions, educational 
levels, and geographic and demographic categories. To maximize the relevance of the 
policymakers’ input, given the constraints of a limited project budget, IES decided on a 
purposive sample. In employing this approach, the task team worked to select the most 
influential policymaking entities while covering the U.S. public education systems, 
including different jurisdictions, educational levels, and regional and demographic 
characteristics. 

Purposive sampling is not meant to produce quantitative information that represents the 
population through statistical estimation. The largely qualitative information collected 
from this purposive sample may nevertheless cover policymakers’ perspectives with a 
reasonable depth and inclusiveness. 

While comprehensive information about the policymaker population is not available, it is 
possible to group the policymaking agencies with approximate counts. The population 
covered by this project included the following groups: 

• Superintendents and other local education officials; 

• Chief state school officers (CSSOs); 

• State higher education executive officers (SHEEOs); 

• State legislators; 

• Governors’ educational policy advisors (GEPAs); 

• Congressional staff members (including staff members of the Senate Committee 
on Health, Education, Labor, and Pensions and the House Committee on 
Education and the Workforce, and educational policy specialists on the staffs of 
members of those committees); and 

• Education association executive directors. 

The total population from which the sample was chosen was estimated to be 
approximately 24,872 policymakers at various levels (local, state, and national) and 
functions (executive, legislative, professional, and advisory). Appendix table A- 1 
presents the counts of people by the decision-making levels and functions, with 
corresponding sample sizes planned for the interview. 
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Table A-l. — Sampled groups of education policymakers 



Stratum 


Estimated 
number in 
universe 


Selected 
number in 
sample 


TOTAL 


24,872 


79 


Superintendents and other local education officials 


17,000 


34 


Chief state school officers (CSSOs) 


51 


10 


State higher education executive officers (SHEEOs) 


51 


10 


State legislators 


7,403 


2 


Governors’ education policy advisors (GEPAs) 


51 


5 


Congressional staff members 


*171 


*8 


Education association executive directors 


245 


10 



* The estimated number of Congressional staff members included, for the Senate, the 52 staff members of 
the Committee on Health, Education, Labor, and Pensions and the education staff specialists for the 27 
members of this committee, and, for the House of Representatives, the 49 staff members of the Committee 
on Education and the Workforce and the education staff specialists for the 21 members of this committee. 
The sample included 2 members of each of these four groups. 



The purposive sampling process involved making judgments using different information 
sources for different groups of policymakers. 

In selecting state- level policymakers (i.e., the CSSOs, SHEEOs, state legislators, and 
GEPAs), a number of issues were considered, including Census regions (Northeast, 
Midwest, South, and West), state population, and state academic performance levels. 
Policymakers from diverse educational perspectives were included in the sample: local 
elementary/secondary officials — the chief state school officers (CSSOs), postsecondary 
officials — the state higher education executive officers (SHEEOs), state legislators, and 
governors’ educational policy advisors (GEPAs). 

A total of 20 CSSOs and SHEEOs were selected from states with large (more than 10 
million), medium (5-10 million), and small (5 million or less) populations across the four 
U.S. Census regions. 

Two state legislators were chosen from the officers or committees of the National 
Conference of State Legislatures. They were the chair or one of the vice-chairs of the 
Assembly on State Issues (ASI), Education Committee, and the Assembly on Federal 
Issues (AFI), Education, Labor, and Workforce Development Committee. States that 
were already represented through CCSO, SHEEO, or GEPA were not selected from this 
group. 

Five GEPAs were selected based on state population size and Census region. The group 
included two large states and three small states that were not selected for the chief state 
school officer or state higher education executive officer samples. Small states where the 
CSSO was also the GEPA were not included. 
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The sample included four legislative assistants for education to selected members of 
Senate Committee on Health, Education, Labor, and Pensions and House Committee on 
Education and the Workforce. The senior majority and minority education policy 
specialist for both the Senate and House Committees were chosen, as well. 

To represent policymakers in entities other than state and federal governments, a total of 
20 executives from national education associations focused on diverse aspects of 
education policymaking were selected for interviews. The selection was made to include 
a wide array of operational features of the education enterprise nationwide, including 
administrative levels (school, district, and state); educational levels (K-12, community 
colleges, adult education, higher education); ownership or sector (public, private, state, 
city, county, and charter school); special populations served (African American, 
Hispanics); and varied decision making roles (professionals, school boards, trustees, and 
administrators). The selection of participants was informed by consultation with experts 
knowledgeable about the complexity of education decision making systems and 
processes. 

As noted above, purposive sampling was the approach chosen in order to focus limited 
resources on gathering input from diverse groups of policymakers. In our judgment, the 
selected education policymakers typify the education policy community in opinions 
regarding education research. While the sample does not warrant statistical generalization 
of the findings to a national population, we believe the information collected nevertheless 
offers insights for IES to shape new research priorities. 

Local- and state- level policymakers were selected so that states and localities were 
represented across key geographic and demographic categories, including: Census region, 
district urban- rural locale, district enrollment size, state percent of urban population, state 
population size, state math 4 th grade achievement level, and within- state achievement 
level. Information sources used included the Census Bureau’s 2002 population estimate, 
the Common Core of Data (CCD) on district enrollment and locale, the 2000 NAEP 
mathematics performance by states, and district average scores on state achievement or 
performance tests. See appendix tables A-2 and A-3 for a summary of characteristics of 
the selected districts and states. 

For the 79 individuals selected in the sample, 71 interviews were completed, achieving a 
90 percent response rate. See appendix table A-4 for a breakdown of the number of 
individuals in the sample and the number of respondents by the seven types of education 
policymakers interviewed. 
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Table A-2. — Numbers of sampled districts and states, by specified demographic and 
geographic characteristics 



District characteristics 1 


No. of district 
policymakers 

34 State characteristics 


No. of state 
policymakers 
27 


Locale 


Urban population 4 




Large central city 


8 More than 85 percent 


7 


Mid-size central city 


3 4 51-85 percent 


4 16 


Urban fringe of large city 


7 50 percent or less 


4 


Small town 


5 




Rural, outside MSA 


7 Achievement 5 




Rural, inside MSA 


3 At national average 


6 18 




Higher than average 


4 




Lower than average 


5 


Region 


Region 7 




Midwest 


1 1 Midwest 


8 


Northeast 


9 Northeast 


5 


South 


8 8 South 


9 


West 


6 West 


5 


Enrollment size 


Population size 9 




Large (>30,000) 


1 0 Large (> 1 0 million) 


11 


Medium (1,000-30,000) 


1 5 Medium (5- 1 0 million) 


i° 8 


Small (<1,000) 


9 Small (< 5 million) 


8 



1 NCES 2000 Common Core of Data: District Locator, see http://nces.ed.gov/ccd/districtsearch 

2 Percent of state population living in metropolitan areas, from table No. 30 in U.S. Bureau of the Census, 
Statistical Abstract of the United States, 2001. 

3 One district policymaker from a mid-size central city was replaced by a district policymaker from a large 
central city. 

4 One policymaker from a state with 51-85 percent urban population was replaced by a policymaker from a 
state with more than 85 percent urban population. 

5 NAEP Math 2000 4th grade, states compared with national average, see 
http://nces.ed.gov/nationsreportcard/mathematics/results/stateachieve-g4.asp 

6 One policymaker from a state with an achievement level at the national average was replaced by a 
policymaker from a state where the achievement level was higher than the national average. 

7 Census regions and divisions, see http://eire.census.gov/popest/geographic/estimatesgeography.php 

8 One policymaker from a district in the South was replaced by a policymaker from a district in the West. 

9 Census population estimates, July 1, 2001, see 

http//eire. census. go v/popest/data/states/populartables/tableOl.php (small: less than 5 million; medium: 5 
million to 10 million; large: more than 10 million). 

10 One policymaker from a medium-sized state was replaced by a policymaker from a small-sized state. 
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Table A-3. — Selected states, number of sampled individuals, and state 
characteristics 



State 


Number of sampled 
individuals 




State characteristics 




State District 
policymakers officials 


Region 1 


Size 2 


Achieve- 

ment 3 


Percent 

urban 4 


TOTAL 


27 


34 










Ohio 


1 


3 


Midwest 


Large 


Average 


81 


Illinois 


2 


3 


Midwest 


Large 


Average 


85 


Indiana 


1 


3 


Midwest 


Medium 


Higher 


72 


Michigan 


1 


0 


Midwest 


Medium 


Average 


82 


Wisconsin 


5 1 


0 


Midwest 


Medium 


Average 


68 


North Dakota 


1 


2 


Midwest 


Small 


Average 


44 


Minnesota 


1 


0 


Midwest 


Small 


Higher 


70 


New York 


1 


3 


Northeast 


Large 


Average 


92 


Pennsylvania 


1 


3 


Northeast 


Large 


Average 


85 


Massachusetts 


1 


3 


Northeast 


Medium 


Higher 


96 


New Jersey 


1 


0 


Northeast 


Medium 


Average 


100 


Connecticut 


1 


0 


Northeast 


Small 


Higher 


96 


Florida 


1 


0 


South 


Large 


Average 


93 


Texas 


6 3 


3 


South 


Large 


Average 


85 


Georgia 


0 


3 


South 


Medium 


Lower 


69 


North Carolina 


1 


0 


South 


Medium 


Average 


68 


Virginia 


1 


0 


South 


Medium 


Average 


78 


Alabama 


1 


0 


South 


Small 


Lower 


70 


Kentucky 


0 


7 1 


South 


Small 


Lower 


49 


Louisiana 


1 


1 


South 


Small 


Lower 


75 


Mississippi 


1 


0 


South 


Small 


Lower 


36 


California 


2 


3 


West 


Large 


Lower 


97 


Washington 


1 


0 


West 


Medium 


Average 


83 


Wyoming 


1 


3 


West 


Small 


Average 


30 


Idaho 


1 


0 


West 


Small 


Average 


39 



2 Census population estimates, July 1 , 200 1 , see 

http//eire. census. gov/popest/data/states/populartables/tableOl.php (small: less than 5 million; medium: 5 
million to 10 million; large: more than 10 million). 

3 NAEP Math 2000 4th grade, states compared with national average, see 
http://nces.ed.gov/nationsreportcard/mathematics/results/stateachieve-g4.asp 

4 Percent of state population living in metropolitan areas, from table No. 30 in U.S. Bureau of the Census, 
Statistical Abstract of the United States, 2001. 

5 The state policymaker from Wisconsin was replaced by a state policymaker from Minnesota. 

6 One of the Texas state policymakers was replaced by a state policymaker from Florida. 

7 

The district official from Kentucky was replaced by a district official from New Mexico. New Mexico is a 
small size state in the South region, and in the lower achievement range; its population is 60 percent urban. 



BEST COPY AVAILABLE 



o 

ERIC 



IES Findings from Interviews with Education Policymakers 



Appendix A 



8.1 



Synectics for Management Decisions, Inc. 



A -6 



Table A-4. — Number of sampled individuals and completed interviews, by 



policymaker group 



Policymaker group 


Number of 
sampled 
individuals 


Number of 
completed 
interviews 


Response 

rate 

(percent) 


TOTAL 


79 


71 


90 


Superintendents and other local education officials 


34 


30 


88 


Chief state school officers (CSSOs) 


10 


9 


90 


State higher education executive officers (SHEEOs) 


10 


10 


100 


State legislators 


2 


2 


100 


Governors’ education policy advisors (GEPAs) 


5 


4 


80 


Congressional staff members 


8 


6 


75 


Education association executive directors 


10 


10 


100 



Notes on Identifying and Categorizing Research Priority Issues 

It was important in this project to identity and communicate accurately the most critical 
concerns of our respondents on their high priorities for research. We needed to categorize 
and consolidate the large amount of information gathered from interviews. After 
completing the interviews, we took a number of steps to analyze and represent the data. 
First, the two interviewers and a senior editor met to develop an approach to analyzing 
and presenting information. We decided to organize the report following the interview 
protocol and reached some consensus on basic categorization of research priority issues. 
The two interviewers would be jointly responsible for documenting the interviews, 
analyzing the information, and drafting the report. They divided the labor by subgroups 
of respondents, with one interviewer working on the local decision makers — the largest 
subgroup — and the other on the remaining respondents. The two interviewers 
systematically reviewed the responses across the subgroups of policymakers. Throughout 
the process, they maintained close communication and developed categories for research 
priority issues for presenting the diverse and spontaneous opinions documented in their 
notes. To ensure consistent priority issues — and to keep them in line with other parts of 
the report — they frequently shared and reviewed each other’s drafts during the report 
writing process. However, no formal coding system was used to present the responses 
regarding priority issues. 
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APPENDIX B 
INTERVIEW PROTOCOL 



Introduction 

Hello, this is , calling on behalf of the U.S. Department of Education. The 

department would very much like to get the advice of <name of respondents about education 
research priorities for the next few years. Is <he/she> available to speak to now for about 15 
minutes? 

Additional explanation 

<If asked “What is this about ? 99 or something similar> 

The U.S. Department of Education is considering what areas of research to emphasize over the 
next few years. The department needs the views of <name of respondents to make sure that the 
research it funds is responsive to the needs of education decision-makers. 

Setting appointment 

<If respondent not available immediately> 

Could you put me on <Dr./Mr./Ms. last name of respondents’s schedule for about 15 minutes 
later today or in the next day or two, or could you tell me a good time today or tomorrow to call 
back to speak to <him/her>? 

<If not available in the next two days> 

What is the earliest time you could put me on <Dr ./Mr./Ms. last name of respondents’s schedule 
for about 15 minutes or that I could call back and speak to <him/her>? 

<Record appointment time and confirm or record call-back time> 

Introduction to target respondent 

<When target respondent is reached> 

The Office of Educational Research and Improvement, or OERI, is trying to make federally- 
funded research more responsive to the needs of education decision-makers. To gain the 
perspectives of education decision-makers in this critical effort, OERI is directly contacting a 
small group of education policy leaders across the United States. We hope that you will be able to 
share with us, for no more than 15 minutes, some of your thoughts on priorities for educational 
research in the next few years. Your thoughts will assist OERI in determining its research and 
funding priorities. 
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Interview 

1. What, in your opinion, are the two highest priority areas in which further research is most 
needed? 

(a) Why is this so? 

(b) <Be sure we know what school level the respondent is addressing and provide 
the respondent the opportunity to think outside K-12. If this is not clear , add:> 

And, when you describe these topics, what level of education are you mainly 

concerned with? 

(c) Of the priority areas you listed, which would you rank higher? 

<If the respondent has difficulty coming up with research areas , suggest the following as 
illustrative :> student achievement, effective instructional practices, dropouts and college 
completion, teacher and school quality, school funding, and equal opportunities for all 
children 

Now, I would like to ask you a couple of questions about your use of education research and 

how accessible that research is. 

2. When looking for information on effective educational programs or practices, do you 
read research studies or reports of evaluations of the programs you are interested in (a) 
never, (b) only some of the time, (c) most of the time, or (d) just about always? 

<If the respondent indicates b, c, ord f then ask questions (a) and (b) below , otherwise 
skip to question (b) below> 

(a) When you have used research information, how have you obtained it? <If the 
respondent needs to be prompted , the interviewer can say> For example, from 
colleagues; newsletters or reports from professional associations; research reports 
and summaries available through journals, ERIC, ED Week or other print and 
online media? 

(b) What would make it easier for you to use research information on a regular 
basis? 

3. What could the U.S. Department of Education do to make education research more 
useful, more accessible, or relevant to your work? 

4. In terms of your own work in education, what are your major policy interests? 

(a) Has the research you’ve used been useful to you in addressing your specific areas 
of interest or providing fruitful guidance? 

(b) Could you tell me about your sense of the research you’ve used, both in terms of 
the amount of existing research and the quality of that research? <Prompt for 
explanations. If respondent needs a prompt , add:> Relevant factors might 
include coverage of the issue or area, number of studies available, their relevance 
to current practice, the strength of their methodology and evidence, ideological or 
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issue biases, accuracy of reporting, extent of dissemination, and adequacy of 
funding? 

5. Finally, on reflection, are there any other high priority issues, areas, or themes in 

American education in which you would like to see more, better, or a different type of 
research? 



O 



IES Findings from Interviews with Education Policymakers 



Appendix B 



Synectics for Management Decisions , Inc. 



C-l 



APPENDIX C 

SAMPLE INTRODUCTORY LETTER 

September 1 1 , 2002 
[Address] 

Dear ... : 

I am writing to ask you to participate in a survey on your education research interests, 
priorities for education research, and opinions about the current state of education 
research in those areas. The findings will help the Office of Educational Research and 
Improvement (OERI), U.S. Department of Education, to develop and promote research 
programs that meet the needs of the education community and those who help shape 
education issues. 

The Department’s Strategic Plan for 2002 — 2007 calls for increasing the relevance of 
education research to meet the needs of our customers. As part of this effort, OERI will 
periodically conduct fast- response surveys of education decision-makers to help 
determine the issues that concern them and about which they need information. To assess 
the state of education research in the U.S., the satisfaction levels of its consumers, and as 
an aid in developing OERI’s priorities, we are asking a select group of education 
decision-makers to take part in a survey. 

Over the next few weeks, Synectics for Management Decisions, a research firm under 
contract to OERI, will be calling you. I would greatly appreciate your taking about 15 to 
20 minutes to talk to them. The findings will be compiled in a report to me. The report 
will not identify respondents. Synectics will not provide the notes of their conversations 
to my office. Since we have selected only 30 state- level policymakers, 29 Congressional 
staff members, and 20 directors of education associations, your participation is very 
important to ensure that the report is representative of the education leadership 
community. 

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a 
collection of information unless such collection displays a valid OMB control number. The 
valid OMB control number for this information collection is 1800-0011 . The time required 
to complete this information collection is estimated to average 20 minutes per response. If 
you have any comments concerning the accuracy of the time estimate(s) or suggestions 
for improving the survey, please write to: U.S. Department of Education, Washington, 

D.C. 20202-4651. If you have comments or concerns regarding the status of your 
response to this survey, write directly to: John Ralph, National Center for Education 
Statistics, U.S. Department of Education, 1990 K Street, NW, Room 9037, Washington, 

D.C. 20006. 

Synectics’ staff will be happy to answer any questions you might have about this activity 
when they call. If you have questions in advance, you may also contact Dr. Sameena 
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Salvucci at Synectics, who is coordinating the study, at 703.807.2309 or e-mail her at 
sams@smdi.com. 

I am looking forward to hearing your views distilled in this report and sincerely hope that 
you will participate in this effort. 



Sincerely, 




Grover J. Whitehurst 

Assistant Secretary of Education for 

Educational Research and Improvement 



O 

ERIC 
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APPENDIX D 

COUNTS BY FREQUENCY OF READING RESEARCH 

REPORTS 



Table D-l. — Counts by frequency of reading research reports 



Stratum 


Only 
some of 
Never the time 


Most 
of the 
time 


Just 

about 

always 


Total 


TOTAL 


2 


20 


20 


29 


71 


Superintendents and other local education 


officials 


1 


5 


8 


16 


30 


CSSOs 


1 


4 


2 


2 


9 


SHEEOs 


0 


5 


3 


2 


10 


State legislators 


0 


2 


0 


0 


2 


Governors’ education policy advisors 


0 


0 


1 


3 


4 


Congressional staff members 


0 


1 


2 


3 


6 


Education association executive directors 


0 


3 


4 


3 


10 



Note: Some respondents gave more detailed description of their level of research use than a simple 
response by the five categories; thus some of the counts of the frequency categories were based on derived 
information. 
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